Simon Willison’s Weblog

Subscribe

Weeknotes: Plugins for LLM, sqlite-utils and Datasette

5th August 2023

The principle theme for the past few weeks has been plugins.

Llama 2 in LLM via plugins

I added the ability to support models other than the OpenAI ones to my LLM command-line tool last month. The timing on this could not have been better: Llama 2 (the first commercially usable version of Meta’s LLaMA language model) was released on July 18th, and I was able to add support to prompting it via LLM that very morning thanks to the llm-replicate plugin I had released the day before that launch.

(I had heard a tip that a new exciting LLM was about to be released on Replicate, though I didn’t realize it was Llama 2 until after the announcement.)

A few days ago I took that a step further: the new llm-llama-cpp plugin can now be used to run a GGML quantized version of the Llama 2 model directly on your own hardware.

LLM is available in Homebrew core now, so getting Llama 2 working is as simple as:

brew install llm
llm install llm-llama-cpp llama-cpp-python
llm llama-cpp download-model \
  https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q8_0.bin \
  --alias llama2-chat --alias l2c --llama2-chat

Then:

llm -m l2c 'Tell me a joke about a llama'

I wrote more about this in Run Llama 2 on your own Mac using LLM and Homebrew—including instructions for calling Llama 2 using the LLM Python API as well.

Plugins for sqlite-utils

My sqlite-utils project, similar to LLM, is a combined CLI tool and Python library. Based on requests from the community I adding plugin support to it too.

There are two categories of plugins so far: plugins that add extra commands to the sqlite-utils CLI tool, and plugins that add extra custom SQL functions that can be executed against SQLite.

There are quite a few plugins listed in the sqlite-utils plugins directory already.

I built sqlite-utils-shell in time for the initial launch, to help demonstrate the new system by adding a sqlite-utils shell command that opens an interactive shell enabling any SQL functions that have been installed by other plugins.

Alex Garcia suggested I look at litecli by Amjith Ramanujam, which is a much more sophisticated terminal shell for SQLite, incorporating auto-completion against tables and columns.

I used that to build a better alternative to my sqlite-utils-shell plugin: sqlite-utils-litecli, which lets you run the following command to get a full litecli shell with all of the custom SQL functions from other plugins:

sqlite-utils litecli mydatabase.db

Screenshot showing the plugin in action - it includes autocomplete of SQLite table names

datasette-auth-tokens and dclient

Meanwhile, in Datasette land... I’ve been investing more time building Datasette Cloud, the SaaS cloud hosted version of Datasette.

The Datasette 1.0 alphas introduced a write API. I wanted a mechanism for Datasette Cloud users to be able to setup automatic imports of data into their instances, taking advantage of that API.

This meant I needed an API key mechanism that allowed tokens to be both created and revoked interactively.

I ended up building that into the existing datasette-auth-tokens plugin, released in preview in the datasette-auth-tokens 0.4a0 alpha.

I’ve been quietly working on a new CLI utility for interacting with Datasette instances via the API, called dcloud. I shipped dcloud 0.2 with a new dclient insert command that can read CSV, TSV or JSON data and write it to an external Datasette instance using that new 1.0 write API.

I’ll have more news to share about Datasette Cloud soon!

Large Language Model talk at North Bay Python

On Sunday I gave the closing talk at North Bay Python, titled Catching up on the weird world of LLMs.

I tried to summarize the last few years of development in the field of LLMs in just 40 minutes. I’m pretty happy with how it turned out! I’ve since published a full annotated transcript of the talk, with slides, additional links and notes—so even if you don’t want to watch the full talk you can still read through a thorough summary of what I covered.

I’ve given a few of my talks this treatment now and I really like it—it’s a great way to unlock as much value as possible from the time I spend putting one of these things together.

Examples of this format:

This time round I built a small tool to help me assemble the notes and alt attributes for the video—I hope to write more about that soon.

Blog entries these weeks

Releases these weeks

TIL these weeks

Posted 5th August 2023 at 12:32 am · Follow me on Mastodon or Twitter or subscribe to my newsletter

More recent articles

This is Weeknotes: Plugins for LLM, sqlite-utils and Datasette by Simon Willison, posted on 5th August 2023.

Next: How I make annotated presentations

Previous: Catching up on the weird world of LLMs