Plugin support for Datasette Lite
I’ve added a new feature to Datasette Lite, my distribution of Datasette that runs entirely in the browser using Python and SQLite compiled to WebAssembly. You can now install additional Datasette plugins by passing them in the URL.
Datasette Lite background
Datasette Lite runs Datasette in the browser. I initially built it as a fun technical proof of concept, but I’m increasingly finding it to be a genuinely useful tool for quick ad-hoc data analysis and publication. Not having any server-side components at all makes it effectively free to use without fear of racking up cloud computing costs for a throwaway project.
You can read more about Datasette Lite in these posts:
- Datasette Lite: a server-side Python web application running in a browser
- Joining CSV files in your browser using Datasette Lite
- Scraping data into Datasette Lite shows an example project where I scraped PSF board resolutions, stored the results in a CSV file in a GitHub Gist and then constructed this URL to open the result in Datasette Lite and execute a SQL query.
Adding plugins to Datasette Lite
One of Datasette’s key features is support for plugins. There are over 90 listed in the plugin directory now, with more emerging all the time. They’re a fantastic way to explore new feature ideas and extend the software to handle non-default use cases.
Plugins are Python packages, published to PyPI. You can add them to Datasette Lite using the new
?install=name-of-plugin query string parameter.
Here’s an example URL that loads the datasette-jellyfish plugin, which adds new SQL functions for calculating distances between strings, then executes a SQL query that demonstrates that plugin:
That URL uses
?install=datasette-jellyfish to install the plugin, then executes the following SQL query:
SELECT levenshtein_distance(:s1, :s2), damerau_levenshtein_distance(:s1, :s2), hamming_distance(:s1, :s2), jaro_similarity(:s1, :s2), jaro_winkler_similarity(:s1, :s2), match_rating_comparison(:s1, :s2);
"barack obama" and
"barrack h obama".
Unfortunately, many existing Datasette plugins aren’t yet compatible with Datasette Lite. Most importantly, visualization plugins such as datasette-cluster-map and datasette-vega don’t work.
Here’s the full list of plugins that I’ve confirmed work with Datasette Lite so far:
- datasette-packages—Show a list of currently installed Python packages—demo
- datasette-dateutil—dateutil functions for Datasette—demo
- datasette-schema-versions—Datasette plugin that shows the schema version of every attached database—demo
- datasette-debug-asgi—Datasette plugin for dumping out the ASGI scope.—demo
- datasette-query-links—Turn SELECT queries returned by a query into links to execute them—demo
- datasette-json-html—Datasette plugin for rendering HTML based on JSON values—demo
- datasette-haversine—Datasette plugin that adds a custom SQL function for haversine distances—demo
- datasette-jellyfish—Datasette plugin that adds custom SQL functions for fuzzy string matching, built on top of the Jellyfish Python library—demo
- datasette-pretty-json—Datasette plugin that pretty-prints any column values that are valid JSON objects or arrays.—demo
- datasette-yaml—Export Datasette records as YAML—demo
- datasette-copyable—Datasette plugin for outputting tables in formats suitable for copy and paste—demo
How it works
The implementation is pretty simple—it can be seen in this commit. The short version is that
?install= options are passed through to the Python web worker that powers Datasette Lite, which then runs the following:
for install_url in install_urls: await micropip.install(install_url)
micropip is a component of Pyodide which knows how to install pure Python wheels directly from PyPI into the browser’s emulated Python environment. If you open up the browser devtools networking panel you can see that in action!
?install= parameter is being passed directly to
micropip.install() you don’t even need to provide names of packages hosted on PyPI—you could instead provide the URL to a wheel file that you’re hosting elsewhere.
This means you can use
?install= as a code injection attack—you can install any Python code you want into the environent. I think that’s fine—the only person who will be affected by this is the user who is viewing the page, and the
lite.datasette.io domain deliberately doesn’t have any cookies set that could cause problems if someone were to steal them in some way.
More recent articles
- ChatGPT should include inline tips - 30th May 2023
- Lawyer cites fake cases invented by ChatGPT, judge is not amused - 27th May 2023
- llm, ttok and strip-tags - CLI tools for working with ChatGPT and other LLMs - 18th May 2023
- Delimiters won't save you from prompt injection - 11th May 2023
- Weeknotes: sqlite-utils 3.31, download-esm, Python in a sandbox - 10th May 2023
- Leaked Google document: "We Have No Moat, And Neither Does OpenAI" - 4th May 2023
- Midjourney 5.1 - 4th May 2023
- Prompt injection explained, with video, slides, and a transcript - 2nd May 2023
- download-esm: a tool for downloading ECMAScript modules - 2nd May 2023
- Let's be bear or bunny - 1st May 2023