Simon Willison’s Weblog

Weeknotes: Datasette 0.43

29th May 2020

My main achievement this week was shipping Datasette 0.43, with a collection of smaller improvements and one big one: a redesign of the register_output_renderer plugin hook.

Generating Atom and iCal feeds with Datasette

The register_output_renderer hook was contributed by Russ Garrett last year. It added a mechanism for plugins to create new output formats for Datasette.

Output formats are controlled by their file extension. Out of the box Datasette supports .json and .csv formats. The hook allows plugins to register new ones—.atom for Atom feeds and .ics for iCal feeds, for example.

I built those exact two plugins using the hook: datasette-atom and datasette-ics. In building them I ran into a few limitations. Atom feeds need to reflect the full URL of the feed for example, but that information wasn’t made available through the plugin hook.

The pattern I settled on for both of my plugins was to require SQL queries that produced data in a specific shape. datasette-atom for example requires a SQL query that returns columns atom_id, atom_title and atom_updated—plus optional columns atom_content, atom_link and a few others.

If those columns are present in the query, the plugin can render an Atom feed! If the columns are not present, it returns an error.

The problem was that EVERY page on the site that could return .json and .csv now got an .atom link as well—even if that link wouldn’t actually work.

So I’ve added a new can_render callback to the plugin registry, which indicates if the that link should be displayed. I shipped a new release, datasette-atom 0.6, that takes advantage of this new feature.

You can see it in action in the Atom feed for my Niche Museums site, which finally passes the Feed Validator!

Also in Datasette 0.43

Copied from the release notes:

• Visually distinguish float and integer columns—useful for figuring out why order-by-column might be returning unexpected results. (#729)
• The Request object, which is passed to several plugin hooks, is now documented. (#706)
• New metadata.json option for setting a custom default page size for specific tables and views, see Setting a custom page size. (#751)
• Canned queries can now be configured with a default URL fragment hash, useful when working with plugins such as datasette-vega, see Setting a default fragment. (#706)
• Fixed a bug in datasette publish when running on operating systems where the /tmp directory lives in a different volume, using a backport of the Python 3.8 shutil.copytree() function. (#744)
• Every plugin hook is now covered by the unit tests, and a new unit test checks that each plugin hook has at least one corresponding test. (#771, #773)

TIL this week

• Introspecting Python function parameters
• Asserting a dictionary is a subset of another dictionary