Weeknotes: Datasette 0.33
23rd December 2019
I released Datasette 0.33 yesterday. The release represents an accumulation of small changes and features since Datasette 0.32 back in November. Duplicating the release notes:
rowidis now included in dropdown menus for filtering tables (#636)
- Columns are now only suggested for faceting if they have at least one value with more than one record (#638)
- Queries with no results now display “0 results” (#637)
- Improved documentation for the
- asyncio task information is now included on the
- Bumped Uvicorn dependency 0.11
- You can now use
--port 0to listen on an available port
template_debugsetting for debugging templates, e.g. https://latest.datasette.io/fixtures/roadsideattractions?context=1 (#654)
The last three items deserve extra explanation.
I run a lot of Datasette instances on my laptop—I often have 5 or 6 running representing different projects or experiments.
Eventually this means I start to hit port conflicts. I’ll run
datasette mydb.db -p 8005 and get an error because port 8005 is already in use by something else.
I asked on Twitter if there were any downsides to having Datasette automatically pick an unused port... and @davids and @dominicrodger pointed out that port 0 is a Unix convention for “pick a port for me”.
It turned out Datasette already supported this! If you ran
datasette --port=0 the underlying Uvicorn server would indeed pick an unused port and start serving on it.
There was just one catch: the URL output to the console would show port 0, so it wasn’t clear which port to actually visit. This was a bug in Uvicorn.
Open source to the rescue... I filed an issue with Uvicorn and then constructed a pull request. Tom Christie merged it five hours later and shipped a release. Datasette 0.33 depends on that updated version of Uvicorn, so
--port=0 now works as expected!
I’m always looking for new ways to add debugging and development tools to Datasette. I added support for ?_trace=1 to see executed SQL queries back in May for example.
While thinking through a refactoring of Datasette’s template rendering to better support plugins, I realized there was an opportunity for better debugging around templates.
Datasette supports custom templates, which means template authors need a clear way to see what variables are available to their template logic.
I tried adding a
?_context=1 parameter to dump out the template context, and it instantly helped me better understand what was going on.
There was just one problem: what about secret information? By default Datasette doesn’t include any secrets (signing keys, API tokens etc) in its template context—but Datasette plugins are allowed to add things to the context, and I can’t guarantee that a future plugin won’t add something that shouldn’t be exposed by the
?_context=1 debugging parameter.
The solution was to have context debugging only available if the new
template_debug configuration setting is turned on:
$ datasette --memory --config template_debug:1
I’ve set up the latest.datasette.io demo to run with this setting, so you can now view the context for any page on that site, like so:
Natalie and I are in the UK this week. We spent a few days in London and managed to get in a few niche museums, including the first four on this week’s new listings on Niche Museums:
- Mail Rail in London
- Museum of Funeral History in London
- London Mithraeum in London
- Bank of England Museum in London
- Reserva Cerro Ancón in Panama City
- Buckfast Abbey in Devon
- Dennis Severs’ House in London
I also made a small fix to the site’s Atom feed: it now respects newlines in the entries, by rendering the description using Markdown.
Here’s the implementation. Since the Atom feed is defined by a SQL query (using datasette-atom) I took advantage of Datasette’s ability to load custom plugins from a per-site
plugins/ directory and added a custom SQLite SQL function for rendering markdown, then added that function call to the query that defines the feed.
More recent articles
- Datasette Enrichments: a new plugin framework for augmenting your data - 1st December 2023
- llamafile is the new best way to run a LLM on your own computer - 29th November 2023
- Prompt injection explained, November 2023 edition - 27th November 2023
- I'm on the Newsroom Robots podcast, with thoughts on the OpenAI board - 25th November 2023
- Weeknotes: DevDay, GitHub Universe, OpenAI chaos - 22nd November 2023
- Deciphering clues in a news article to understand how it was reported - 22nd November 2023
- Exploring GPTs: ChatGPT in a trench coat? - 15th November 2023
- Financial sustainability for open source projects at GitHub Universe - 10th November 2023
- ospeak: a CLI tool for speaking text in the terminal via OpenAI - 7th November 2023
- DALL-E 3, GPT4All, PMTiles, sqlite-migrate, datasette-edit-schema - 30th October 2023