Weeknotes: Implementing a write API, Mastodon distractions
23rd November 2022
Everything is so distracting at the moment. The ongoing Twitter catastrophe, the great migration (at least amongst most of the people I pay attention to) to Mastodon, the FTX calamity. It’s been very hard to focus!
I’ve been continuing to work on the write API for Datasette that I described previously. I’ve decided that the first release to include that work will also be the first alpha version of Datasette 1.0—you can see my progress towards that goal in the Datasette 1.0a0 milestone.
This alpha will be the first in a sequence of alphas. There’s still a lot more work to do—most notably:
- Refactor Datasette’s HTML templates to exclusively use values that are available in the API (including via a new
?_extra=
mechanism). This will help achieve the goal of having those template contexts officially documented, such that custom template authors can depend on them being stable not changing between dot-releases. - This means some breaking API changes, which need to be documented and stable before 1.0.
- Finalize the design of the plugin hooks for 1.0
- Change how
metadata.json
works—it’s grown a whole bunch of functionality that has nothing to do with metadata, so I’d like to rename it. - Review how authentication and permissions work—there may be some changes I can make here to improve their usability prior to 1.0.
I hope to put out alpha releases quite frequently as the different parts of 1.0 start to come together.
dclient
Designing a good API is difficult if you don’t have anything that uses it! But you can’t build things against an API that doesn’t exist yet.
To help overcome this chicken-and-egg problem, I’ve started a new project: dclient.
dclient
is the Datasette Client—it’s a CLI utility for interacting with remote Datasette instances.
I’m planning to imitate much of the existing sqlite-utils design, which provides a CLI for manipulating local SQLite database files.
Eventually you’ll be able to use dclient
to authenticate with a remote Datasette instance and then do things like pipe CSV files into it to create new tables.
So far it has one, obvious feature: you can use it to run a SQL query against a remote Datasette instance:
dclient query \
https://datasette.io/content \
"select * from news limit 1"
Returns:
[
{
"date": "2022-10-27",
"body": "[Datasette 0.63](https://docs.datasette.io/en/stable/changelog.html#v0-63) is out. Here are the [annotated release notes](https://simonwillison.net/2022/Oct/27/datasette-0-63/)."
}
]
It also supports aliases, so you can create an alias for a database like this:
dclient alias add content https://datasette.io/content
And then run the above query like this instead:
dclient query content "select * from news limit 1"
One fun additional feature: if you install dclient
in the same virtual environment as Datasette itself it registers itself as a command plugin:
datasette install dclient
You can then access its functionality via datasette client
instead:
datasette client query content \
"select * from news limit 1"
A flurry of plugins
I also pushed out a flurry of plugin releases, listed below. Almost all of these are a result of a tiny change to how breadcrumbs work in Datasette 0.63 which turned out to break the display of navigation in a bunch of plugins. Details in this issue—thanks to Brian Grinstead for pointing it out.
Releases this week
-
dclient: 0.1a2—(3 releases total)—2022-11-22
A client CLI utility for Datasette instances -
datasette-graphql: 2.1.2—(37 releases total)—2022-11-19
Datasette plugin providing an automatic GraphQL API for your SQLite databases -
datasette: 0.63.2—(118 releases total)—2022-11-19
An open source multi-tool for exploring and publishing data -
datasette-edit-schema: 0.5.2—(11 releases total)—2022-11-18
Datasette plugin for modifying table schemas -
datasette-indieauth: 1.2.2—(11 releases total)—2022-11-18
Datasette authentication using IndieAuth and RelMeAuth -
datasette-import-table: 0.3.1—(7 releases total)—2022-11-18
Datasette plugin for importing tables from other Datasette instances -
datasette-public: 0.2.1—(3 releases total)—2022-11-18
Make specific Datasette tables visible to the public -
datasette-copyable: 0.3.2—(5 releases total)—2022-11-18
Datasette plugin for outputting tables in formats suitable for copy and paste -
datasette-edit-templates: 0.2—(3 releases total)—2022-11-18
Plugin allowing Datasette templates to be edited within Datasette -
datasette-configure-fts: 1.1.1—(11 releases total)—2022-11-18
Datasette plugin for enabling full-text search against selected table columns -
datasette-socrata: 0.3.1—(5 releases total)—2022-11-18
Import data from Socrata into Datasette -
datasette-ripgrep: 0.7.1—(12 releases total)—2022-11-18
Web interface for searching your code using ripgrep, built as a Datasette plugin -
datasette-search-all: 1.1.1—(9 releases total)—2022-11-18
Datasette plugin for searching all searchable tables at once
TIL this week
More recent articles
- Storing times for human events - 27th November 2024
- Ask questions of SQLite databases and CSV/JSON files in your terminal - 25th November 2024
- Weeknotes: asynchronous LLMs, synchronous embeddings, and I kind of started a podcast - 22nd November 2024