Weeknotes: Datasette 0.33
I released Datasette 0.33 yesterday. The release represents an accumulation of small changes and features since Datasette 0.32 back in November. Duplicating the release notes:
rowidis now included in dropdown menus for filtering tables (#636)
- Columns are now only suggested for faceting if they have at least one value with more than one record (#638)
- Queries with no results now display “0 results” (#637)
- Improved documentation for the
- asyncio task information is now included on the
- Bumped Uvicorn dependency 0.11
- You can now use
--port 0to listen on an available port
template_debugsetting for debugging templates, e.g. https://latest.datasette.io/fixtures/roadsideattractions?context=1 (#654)
The last three items deserve extra explanation.
I run a lot of Datasette instances on my laptop—I often have 5 or 6 running representing different projects or experiments.
Eventually this means I start to hit port conflicts. I’ll run
datasette mydb.db -p 8005 and get an error because port 8005 is already in use by something else.
I asked on Twitter if there were any downsides to having Datasette automatically pick an unused port... and @davids and @dominicrodger pointed out that port 0 is a Unix convention for “pick a port for me”.
It turned out Datasette already supported this! If you ran
datasette --port=0 the underlying Uvicorn server would indeed pick an unused port and start serving on it.
There was just one catch: the URL output to the console would show port 0, so it wasn’t clear which port to actually visit. This was a bug in Uvicorn.
Open source to the rescue... I filed an issue with Uvicorn and then constructed a pull request. Tom Christie merged it five hours later and shipped a release. Datasette 0.33 depends on that updated version of Uvicorn, so
--port=0 now works as expected!
I’m always looking for new ways to add debugging and development tools to Datasette. I added support for ?_trace=1 to see executed SQL queries back in May for example.
While thinking through a refactoring of Datasette’s template rendering to better support plugins, I realized there was an opportunity for better debugging around templates.
Datasette supports custom templates, which means template authors need a clear way to see what variables are available to their template logic.
I tried adding a
?_context=1 parameter to dump out the template context, and it instantly helped me better understand what was going on.
There was just one problem: what about secret information? By default Datasette doesn’t include any secrets (signing keys, API tokens etc) in its template context—but Datasette plugins are allowed to add things to the context, and I can’t guarantee that a future plugin won’t add something that shouldn’t be exposed by the
?_context=1 debugging parameter.
The solution was to have context debugging only available if the new
template_debug configuration setting is turned on:
$ datasette --memory --config template_debug:1
I’ve set up the latest.datasette.io demo to run with this setting, so you can now view the context for any page on that site, like so:
Natalie and I are in the UK this week. We spent a few days in London and managed to get in a few niche museums, including the first four on this week’s new listings on Niche Museums:
- Mail Rail in London
- Museum of Funeral History in London
- London Mithraeum in London
- Bank of England Museum in London
- Reserva Cerro Ancón in Panama City
- Buckfast Abbey in Devon
- Dennis Severs’ House in London
I also made a small fix to the site’s Atom feed: it now respects newlines in the entries, by rendering the description using Markdown.
Here’s the implementation. Since the Atom feed is defined by a SQL query (using datasette-atom) I took advantage of Datasette’s ability to load custom plugins from a per-site
plugins/ directory and added a custom SQLite SQL function for rendering markdown, then added that function call to the query that defines the feed.
More recent articles
- Lawyer cites fake cases invented by ChatGPT, judge is not amused - 27th May 2023
- llm, ttok and strip-tags - CLI tools for working with ChatGPT and other LLMs - 18th May 2023
- Delimiters won't save you from prompt injection - 11th May 2023
- Weeknotes: sqlite-utils 3.31, download-esm, Python in a sandbox - 10th May 2023
- Leaked Google document: "We Have No Moat, And Neither Does OpenAI" - 4th May 2023
- Midjourney 5.1 - 4th May 2023
- Prompt injection explained, with video, slides, and a transcript - 2nd May 2023
- download-esm: a tool for downloading ECMAScript modules - 2nd May 2023
- Let's be bear or bunny - 1st May 2023
- Weeknotes: Miscellaneous research into Rye, ChatGPT Code Interpreter and openai-to-sqlite - 1st May 2023