Simon Willison’s Weblog

Weeknotes: datasette-ephemeral-tables, datasette-export

5th December 2022

Most of what I’ve been working on for the past week and a half is already documented:

• Learning Rust with ChatGPT, Copilot and Advent of Code
• A new AI game: Give me ideas for crimes to do
• The big one: Datasette’s new JSON write API: The first alpha of Datasette 1.0
• And my DjangoCon talk write-up: Coping strategies for the serial project hoarder

I also released two new Datasette plugins—one of them to support the Datasette 1.0 alpha release, and another which is still shaping up.

datasette-ephemeral-tables

For the Datasette 1.0 alpha I wanted a way for people to try out the new write API, without having to install Datasette themselves.

But I didn’t want to commit to running a public writable demo that anyone could use for the long-term.

I decided to solve this using a new plugin. I built datasette-ephemeral-tables, which creates an in-memory SQLite database with tables that are automatically dropped fifteen minutes after they are created.

It’s a pretty fun little plugin. The source code is here—it works by running a task every 2 seconds which scans the ephemeral database for new tables and adds their creation time to a dictionary, then drops any that were created more than X minutes ago.

It also adds a UI element to the table page using JavaScript, which shows a prominent countdown timer to warn you that the table is not long for this world:

I then built the todomvc.datasette.io demo on top of a new ephemeral table hosted by the latest.datasette.io demo instance.

Getting that demo to work involved figuring out CORS for the write API, a very useful new ability which I shipped in the Datasette 1.0a1 alpha release.

datasette-export

The datasette-export plugin is still in early alpha. The idea is to turn Datasette into a static site generator tool, by providing a CLI that can export multiple pages from a Datasette instance directly to static files on disk.

Here’s an example of it in action:

datasette export . \
  --path / \
  --path /plugins \
  --sql "select '/plugins/' || name from content.plugins" \
  --crossdb

This would run against database files, templates and plugins files in the current directory (the . argument) and exports the / page and the /plugins page, then uses a SQL query to specify a list of additional pages and exports those too.

The --crossdb option is necessary because this example project (the datasette.io site) has more than one database, and using that option lets you select ... from content.plugins to specify a particular database.

Plenty more details on what works and what’s coming next for that project in that repository’s issues.

Releases this week

• datasette-public: 0.2.2—(4 releases total)—2022-12-02
  Make specific Datasette tables visible to the public
• datasette-ephemeral-tables: 0.2.2—(4 releases total)—2022-12-02
  Provide tables that expire after a time limit
• datasette: 1.0a1—(120 releases total)—2022-12-01
  An open source multi-tool for exploring and publishing data
• datasette-export: 0.1a0—2022-11-27
  Export pages from Datasette to files on disk

TIL this week

• Lazy loading images in HTML
• actions/setup-python caching for setup.py projects
• Using pipenv and Docker
• Workaround for google-github-actions/setup-gcloud errors