Simon Willison’s Weblog

Using gpt-4o-mini as a reranker. Tip from David Zhang: "using gpt-4-mini as a reranker gives you better results, and now with strict mode it's just as reliable as any other reranker model".

David's code here demonstrates the Vercel AI SDK for TypeScript, and its support for structured data using Zod schemas.

const res = await generateObject({
  model: gpt4MiniModel,
  prompt: `Given the list of search results, produce an array of scores measuring the liklihood of the search result containing information that would be useful for a report on the following objective: ${objective}\n\nHere are the search results:\n<results>\n${resultsString}\n</results>`,
  system: systemMessage(),
  schema: z.object({
    scores: z
      .object({
        reason: z
          .string()
          .describe(
            'Think step by step, describe your reasoning for choosing this score.',
          ),
        id: z.string().describe('The id of the search result.'),
        score: z
          .enum(['low', 'medium', 'high'])
          .describe(
            'Score of relevancy of the result, should be low, medium, or high.',
          ),
      })
      .array()
      .describe(
        'An array of scores. Make sure to give a score to all ${results.length} results.',
      ),
  }),
});

It's using the trick where you request a reason key prior to the score, in order to implement chain-of-thought - see also Matt Webb's Braggoscope Prompts.

# 6:06 pm / ai, prompt-engineering, generative-ai, gpt-4, llms

PEP 750 – Tag Strings For Writing Domain-Specific Languages. A new PEP by Jim Baker, Guido van Rossum and Paul Everitt that proposes introducing a feature to Python inspired by JavaScript's tagged template literals.

F strings in Python already use a f"f prefix", this proposes allowing any Python symbol in the current scope to be used as a string prefix as well.

I'm excited about this. Imagine being able to compose SQL queries like this:

query = sql"select * from articles where id = {id}"

Where the sql tag ensures that the {id} value there is correctly quoted and escaped.

Currently under active discussion on the official Python discussion forum.

# 6:29 pm / guido-van-rossum, javascript, python

Ladybird set to adopt Swift. Andreas Kling on the Ladybird browser project's search for a memory-safe language to use in conjunction with their existing C++ codebase:

Over the last few months, I've asked a bunch of folks to pick some little part of our project and try rewriting it in the different languages we were evaluating. The feedback was very clear: everyone preferred Swift!

Andreas previously worked for Apple on Safari, but this was still a surprising result given the current relative lack of widely adopted open source Swift projects outside of the Apple ecosystem.

This change is currently blocked on the upcoming Swift 6 release:

We aren't able to start using it just yet, as the current release of Swift ships with a version of Clang that's too old to grok our existing C++ codebase. But when Swift 6 comes out of beta this fall, we will begin using it!

# 6:38 pm / browsers, c-plus-plus, andreas-kling, ladybird

Transformer Explainer. This is a very neat interactive visualization (with accompanying essay and video - scroll down for those) that explains the Transformer architecture for LLMs, using a GPT-2 model running directly in the browser using the ONNX runtime and Andrej Karpathy's nanoGPT project.

# 10:56 pm / ai, explorables, d3, generative-ai, llms

Using sqlite-vec with embeddings in sqlite-utils and Datasette. My notes on trying out Alex Garcia's newly released sqlite-vec SQLite extension, including how to use it with OpenAI embeddings in both Datasette and sqlite-utils.

# 11:37 pm / sqlite, datasette, sqlite-utils, openai, alex-garcia, embeddings