Simon Willison’s Weblog

LLM 0.27, the annotated release notes: GPT-5 and improved tool calling

11th August 2025

I shipped LLM 0.27 today (followed by a 0.27.1 with minor bug fixes), adding support for the new GPT-5 family of models from OpenAI plus a flurry of improvements to the tool calling features introduced in LLM 0.26. Here are the annotated release notes.

GPT-5

• New models: gpt-5, gpt-5-mini and gpt-5-nano. #1229

I would have liked to get these out sooner, but LLM had accumulated quite a lot of other changes since the last release and I wanted to use GPT-5 as an excuse to wrap all of those up and get them out there.

These models work much the same as other OpenAI models, but they have a new reasoning_effort option of minimal. You can try that out like this:

llm -m gpt-5 'A letter advocating for cozy boxes for pelicans in Half Moon Bay harbor' -o reasoning_effort minimal

Setting “minimal” almost completely eliminates the “thinking” time for the model, causing it to behave more like GPT-4o.

Here’s the letter it wrote me at a cost of 20 input, 706 output = $0.007085 which is 0.7085 cents.

You can set the default model to GPT-5-mini (since it’s a bit cheaper) like this:

llm models default gpt-5-mini

Tools in templates

• LLM templates can now include a list of tools. These can be named tools from plugins or arbitrary Python function blocks, see Tools in templates. #1009

I think this is the most important feature in the new release.

I added LLM’s tool calling features in LLM 0.26. You can call them from the Python API but you can also call them from the command-line like this:

llm -T llm_version -T llm_time 'Tell the time, then show the version'

Here’s the output of llm logs -c after running that command.

This example shows that you have to explicitly list all of the tools you would like to expose to the model, using the -T/--tool option one or more times.

In LLM 0.27 you can now save these tool collections to a template. Let’s try that now:

llm -T llm_version -T llm_time -m gpt-5 --save mytools

Now mytools is a template that bundles those two tools and sets the default model to GPT-5. We can run it like this:

llm -t mytools 'Time then version'

Let’s do something more fun. My blog has a Datasette mirror which I can run queries against. I’m going to use the llm-tools-datasette plugin to turn that into a tool-driven template. This plugin uses a “toolbox”, which looks a bit like a class. Those are described here.

llm install llm-tools-datasette

# Now create that template
llm --tool 'Datasette("https://datasette.simonwillison.net/simonwillisonblog")' \
  -m gpt-5 -s 'Use Datasette tools to answer questions' --save blog

Now I can ask questions of my database like this:

llm -t blog 'top ten tags by number of entries'

The --td option there stands for --tools-debug—it means we can see all tool calls as they are run.

Here’s the output of the above:

Top 10 tags by number of entries (excluding drafts):
- quora — 1003
- projects — 265
- datasette — 238
- python — 213
- ai — 200
- llms — 200
- generative-ai — 197
- weeknotes — 193
- web-development — 166
- startups — 157

Full transcript with tool traces here.

I’m really excited about the ability to store configured tools

• Tools can now return attachments, for models that support features such as image input. #1014

I want to build a tool that can render SVG to an image, then return that image so the model can see what it has drawn. For reasons.

• New methods on the Toolbox class: .add_tool(), .prepare() and .prepare_async(), described in Dynamic toolboxes. #1111

I added these because there’s a lot of interest in an MCP plugin for Datasette. Part of the challenge with MCP is that the user provides the URL to a server but we then need to introspect that server and dynamically add the tools we have discovered there. The new .add_tool() method can do that, and the .prepare() and .prepare_async() methods give us a reliable way to run some discovery code outside of the class constructor, allowing it to make asynchronous calls if necessary.

• New model.conversation(before_call=x, after_call=y) attributes for registering callback functions to run before and after tool calls. See tool debugging hooks for details. #1088
• Raising llm.CancelToolCall now only cancels the current tool call, passing an error back to the model and allowing it to continue. #1148

These hooks are useful for implementing more complex tool calling at the Python API layer. In addition to debugging and logging they allow Python code to intercept tool calls and cancel or delay them based on what they are trying to do.

• Some model providers can serve different models from the same configured URL—llm-llama-server for example. Plugins for these providers can now record the resolved model ID of the model that was used to the LLM logs using the response.set_resolved_model(model_id) method. #1117

This solves a frustration I’ve had for a while where some of my plugins log the same model ID for requests that were processed by a bunch of different models under the hood—making my logs less valuable. The new mechanism now allows plugins to record a more accurate model ID for a prompt, should it differ from the model ID that was requsted.

• New -l/--latest option for llm logs -q searchterm for searching logs ordered by date (most recent first) instead of the default relevance search. #1177

My personal log database has grown to over 8,000 entries now, and running full-text search queries against it often returned results from last year that were no longer relevant to me. Being able to find the latest prompt matching “pelican svg” is much more useful.

Everything else was bug fixes and documentation improvements:

Bug fixes and documentation

• The register_embedding_models hook is now documented. #1049
• Show visible stack trace for llm templates show invalid-template-name. #1053
• Handle invalid tool names more gracefully in llm chat. #1104
• Add a Tool plugins section to the plugin directory. #1110
• Error on register(Klass) if the passed class is not a subclass of Toolbox. #1114
• Add -h for --help for all llm CLI commands. #1134
• Add missing dataclasses to advanced model plugins docs. #1137
• Fixed a bug where llm logs -T llm_version "version" --async incorrectly recorded just one single log entry when it should have recorded two. #1150
• All extra OpenAI model keys in extra-openai-models.yaml are now documented. #1228