Weeknotes: datasette-export-notebook, PyInstaller packaged Datasette, CBSAs
10th January 2021
What a terrible week. I’ve found it hard to concentrate on anything substantial. In a mostly futile attempt to distract myself from doomscrolling I’ve mainly been building some experimental output plugins, fiddling with PyInstaller and messing around with shapefiles.
Packaged Datasette with PyInstaller
A long running goal for Datasette has been to make it as easy to install as possible—something that’s not particularly straight-forward for applications written in Python, at least in comparison to toolchains like Rust, Go or Deno.
Back in November 2017 Raffaele Messuti suggested using PyInstaller for this. I revisited that issue while looking through open Datasette issues ordered by least recently updated and decided to try it out again—and it worked! Here’s the resulting TIL, and I’ve attached a bundled
datasette macOS binary file to the 0.53 release on GitHub.
There’s one catch: the binary isn’t signed, which means it shows security warnings that have to be worked around if you try to run it on macOS. I’ve started looking into the signing process—I’m going to need an Apple Developer account and to jump through a bunch of different hoops, but it looks like I should be able to get that working. Here’s my issue for that: GitHub Actions workflow to build and sign macOS binary executables—it looks like gon is the missing automation piece I need.
One thing that really impressed me about PyInstaller is the size of the resulting file. On both macOS and Linux it was able to create a roughly 8MB file containing Datasette, all of its dependencies AND a working Python environment. It’s pretty magic!
I wrote this up in detail a couple of days ago: datasette-css-properties is an amusingly weird output plugin that turns the results of a SQL query into CSS custom property definitions which can then be used to style or insert content into the current page.
I’ve opened an issue to take advantage of this optimization in Datasette itself.
This is an idea I’ve been bouncing around for a while, and during a bout of attempted-coup-induced insomnia I decided to sketch out an initial version.
datasette-export-notebook is a plugin that adds an export-to-notebook option to any table or query.
Here’s a live demo. The interface currently looks like this:
Core-based statistical areas are a US government concept used for various statistical purposes. They are essentially metropolitan areas, based on central cities and the commuting area that they sit inside.
I built cbsa.datasettes.com this week to provide an API for looking up a CBSA based on a latitude and longitude point. Here’s a location within San Francisco for example:
San Francisco-Oakland-Berkeley, CA. Add
&_shape=array to the above URL to get a JSON API version.
The data comes from a shapefile published by the Bureau of Transportation Stastics. I’m using shapefile-to-sqlite to import it into a SpatiaLite database, then publishing it to Cloud Run using this GitHub Actions workfow. Full details in the README.
I built this mainly to act as a simple updated example of how to use Datasette and SpatiaLite to provide an API against data from a shapefile. I published a tutorial about doing this for timezones three years ago, but
shapefile-to-sqlite makes it much easier.
Releases this week
Datasette plugin providing instructions for exporting data to Jupyter or Observable
Experimental Datasette output plugin using CSS properties
Python CLI utility and library for manipulating SQLite databases
TIL this week
More recent articles
- Datasette Enrichments: a new plugin framework for augmenting your data - 1st December 2023
- llamafile is the new best way to run a LLM on your own computer - 29th November 2023
- Prompt injection explained, November 2023 edition - 27th November 2023
- I'm on the Newsroom Robots podcast, with thoughts on the OpenAI board - 25th November 2023
- Weeknotes: DevDay, GitHub Universe, OpenAI chaos - 22nd November 2023
- Deciphering clues in a news article to understand how it was reported - 22nd November 2023
- Exploring GPTs: ChatGPT in a trench coat? - 15th November 2023
- Financial sustainability for open source projects at GitHub Universe - 10th November 2023
- ospeak: a CLI tool for speaking text in the terminal via OpenAI - 7th November 2023
- DALL-E 3, GPT4All, PMTiles, sqlite-migrate, datasette-edit-schema - 30th October 2023