Simon Willison’s Weblog

Weeknotes: datasette-leaflet, datasette-plugin cookiecutter upgrades

31st January 2021

This week I shipped Datasette 0.54, sent out the latest Datasette Newsletter and then mostly worked on follow-up projects.

datasette-leaflet

Datasette 0.54 introduced support for JavaScript modules. datasette-leaflet aims to solve a growing problem: there are now three Datasette plugins that use the Leaflet JavaScript mapping library, and all three were loading it in different ways and using different versions.

Those three plugins—datasette-leaflet-freedraw, datasette-leaflet-geojson and datasette-cluster-map—now all depend on datasette-leaflet. This should mean that even if multiple plugins are installed they will all load and use the same copy of the Leaflet library.

There’s just one problem: datasette-leaflet-freedraw uses the Leaflet.FreeDraw module, which it turns out already bundles its own copy of Leaflet. I still need to figure out how to build a version of that package which can take advantage of the shared copy of Leaflet—see open issue #5.

datasette-plugin and cookiecutter tests

One of the tricker aspects of building a Datasette plugin is bundling static assets such as JavaScript and CSS in the plugin to be distributed via PyPI. I know this is hard because I frequently mess up the package_data= section of setup.py and have to ship another release to fix my mistake!

I’ve upgraded the datasette-plugin cookiecutter template to help with this. The template now asks you if you want to create static/ or template/ directories as part of the interactive configuration—if you say yes to either of those it will create the directories for you and configure package_data= to correctly bundle any files contained within them.

This lead to a couple of new TILs: Testing cookiecutter templates with pytest and Conditionally creating directories in cookiecutter.

datasette-export-notebook improvements

datasette-export-notebook is a plugin that adds copy-and-paste instructions for exporting data from Datasette to Jupyter or Observable notebooks.

The “stream all rows” option for Jupyter uses Datasette’s CSV streaming export, which has the downside of losing any type information.

As of issue #8 the copy-and-paste code for Jupyter now uses the dtype= option to specify the correct types.

You can see it in action on this page, where the Jupyter example code now looks like this:

df = pandas.read_csv(
    "https://covid-19.datasettes.com/covid/us_census_county_populations_2019.csv?_stream=on",
    dtype={
        "fips": int,
        "population": int,
    }
)

Releases this week

• datasette-export-notebook: 0.3—2021-01-29
  Datasette plugin providing instructions for exporting data to Jupyter or Observable
• datasette-debug-asgi: 1.0—2021-01-29
  Datasette plugin for dumping out the ASGI scope
• datasette-template-sql: 1.0.2—2021-01-29
  Datasette plugin for executing SQL queries from templates
• datasette-cluster-map: 0.17— 2021-01-29
  Datasette plugin that shows a map for any data with latitude/longitude columns
• datasette-leaflet-geojson: 0.8—2021-01-26
  Datasette plugin that replaces any GeoJSON column values with a Leaflet map.
• datasette-leaflet-freedraw: 0.2.1—2021-01-25
  Draw polygons on maps in Datasette
• datasette-leaflet: 0.2—2021-01-25
  Datasette plugin adding the Leaflet JavaScript library
• datasette: 0.54— 2021-01-25
  An open source multi-tool for exploring and publishing data

TIL this week

• Testing cookiecutter templates with pytest
• Conditionally creating directories in cookiecutter