Simon Willison’s Weblog

Weeknotes: incremental improvements

24th October 2020

I’ve been writing my talk for PyCon Argentina this week, which has proved surprisingly time consuming. I hope to have that wrapped up soon—I’m pre-recording it, which it turns out is much more work than preparing a talk to stream live.

I’ve made bits and pieces of progress on a whole bunch of different projects. Here are my notes on Datasette, plus an annotated version of my other releases-this-week.

Datasette 0.51a0

Datasette’s base_url configuration option is designed to help run Datasette behind a proxy—so you can configure Apache or nginx to proxy /my-datasette/ to a Datasette instance and have every internal link work correctly.

It doesn’t completely work. I gathered all of the bugs with it in a tracking issue, addressed as many of them as I could and released Datasette 0.51a0 as a testing alpha.

if you run Datasette behind a proxy please try out this new alpha and tell me if it works for you! Testing help is requested here.

Also in the alpha:

• New datasette.urls URL builder for plugins, see Building URLs within plugins. (#904)
• Removed --debug option, which didn’t do anything. (#814)
• Link: HTTP header pagination. (#1014)
• x button for clearing filters. (#1016)
• Edit SQL button on canned queries, (#1019)
• --load-extension=spatialite shortcut. (#1028)

Other releases this week

sphinx-to-sqlite 0.1a1 and 0.1a

One of the features I’m planning for the official Datasette website is combined search across issues, commits, releases, plugins and documentation—powered by my Dogsheep Beta search engine.

This means I nead to load Datasette’s documentation into a SQLite database. sphinx-to-sqlite is my new tool for doing that: it uses the optional XML output from Sphinx to create a SQLite table populated with sections from the documentation, since these seem like the right unit for executing search against.

I’m now using this to build a Datasette instance at latest-docs.datasette.io with the latest documentation on every commit.

datasette-cluster-map 0.14 and 0.13

The default marker popup for datasette-cluster-map is finally a human readable window, not a blob of JSON! You can see that in action on the global-power-plants demo.

inaturalist-to-sqlite 0.2.1, pocket-to-sqlite 0.2.1

I tried out the new PyPI resolver and found that it is a lot less tolerant of ~= v.s. >= dependencies, so I pushed out new releases of these two packages.

datasette-json-preview 0.2

I’m using this plugin to preview the new default JSON representation I’m planning for Datasette 1.0. Carl Johnson provided some useful feedback leading to this new iteration, which now looks like this.

github-to-sqlite 2.7

Quoting the release notes:

github-to-sqlite repos command now takes options --readme and --readme-html, which write the README or rendered HTML README into the readme or readme_html columns, respectively. #52

Another feature I need for the Datasette website search engine, described above.

dogsheep-beta 0.9

My personal search engine, described in the latest Datasette Weekly newsletter. This release added facet by date, part of ongoing work on a timeline view.

I also updated it to take advantage of the datasette.client internal API mechanism introduced in Datasette 0.50.

healthkit-to-sqlite 1.0

Another project bumped to 1.0. Only a small bug fix here: this can now import Apple HealthKit data from devices that use languages other than English.

TIL this week

• Writing JavaScript that responds to media queries

I finally upgraded this blog to show recently added “Elsewhere” content (bookmarks and quotations) interspersed with my main entries in mobile view. I worte up this TIL to explain what I did.