Simon Willison’s Weblog

Weeknotes: datasette-jupyterlite, s3-credentials and a Python packaging talk

5th November 2021

My big project this week was s3-credentials, described yesterday—but I also put together a fun expermiental Datasette plugin bundling JupyterLite and wrote up my PyGotham talk on Python packaging.

datasette-jupyterlite

JupyterLite is absolutely incredible: it’s a full, working distribution of Jupyter that runs entirely in a browser, thanks to a Python interpreter (and various other parts of the scientific Python stack) that has been compiled to WebAssembly by the Pyodide project.

Since it’s just static JavaScript (and WASM modules) it’s possible to host it anywhere that can run a web server.

Datasette runs a web server...

So, I built datasette-jupyterlite—a Datasette plugin that bundles JupyterLite and serves it up as part of the Datasette instance.

You can try a live demo here:

Here’s some Python code that will retrieve data from the associated Datasette instance and pull it into a Pandas DataFrame:

import pandas, pyodide
pandas.read_csv(pyodide.open_url(
  "https://latest-with-plugins.datasette.io/github/stars.csv")
)

(I haven’t yet found a way to do this with a relative rather than absolute URL.)

The best part of this is that it works in Datasette Desktop! You can install the plugin using the “Install and manage plugins” menu item to get a version of Jupyter running in Python running in WebAssembly running in V8 running in Chromium running in Electron.

The plugin implementation is just 30 lines of code—it uses the jupyterlite Python package which bundles a .tgz file containing all of the required static assets, then serves files directly out of that tarfile.

Also this week

My other projects from this week are already written about on the blog:

• s3-credentials: a tool for creating credentials for S3 buckets introduces a new CLI tool I built that automates the process of creating new, long-lived credentials that provide read-only, read-write or write-only access to just a single specified S3 bucket.
• How to build, test and publish an open source Python library is a detailed write-up of the 10 minute workshop I presented at PyGotham this year showing how to create a Python library, bundle it up as a package using setup.py, publish it to PyPI and then set up GitHub Actions to test and publish future releases.

Releases this week

• s3-credentials: 0.4—(4 releases total)—2021-11-04
  A tool for creating credentials for accessing S3 buckets
• datasette-notebook: 0.2a0—(4 releases total)—2021-11-02
  A markdown wiki and dashboarding system for Datasette
• datasette-jupyterlite: 0.1a0—2021-11-01
  JupyterLite as a Datasette plugin

TIL this week

• Using VCR and pytest with pytest-recording
• Quick and dirty mock testing with mock_calls