Simon Willison’s Weblog

Creepy crawlies (via) Konstantin Ryabitsev discusses how bad the "background radiation" of abusive crawlers has become from the perspective of git.kernel.org, the official Git repository for the Linux kernel:

TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html.

I worry about this a lot from the perspective of Datasette, which serves a huge number of crawlable web pages.

# 11:08 pm / crawling, git, linux, datasette, ai-ethics

Research acceleration: The view inside OpenAI. Apparently today is RSI day at OpenAI, for Recursive Self-Improvement - I think it's their new AGI. Both this piece and the new essay An Alien Mind (by Chief Scientist Jakub Pachocki) talk about it, and this one doesn't even bother to expand the acronym.

Included are details on how OpenAI's own research team are using coding agents. Like pretty much everyone else 2026 has been the year that agentic engineering really took off at OpenAI, best illustrated by this chart:

I'm intrigued at what caused that significant acceleration in AI spend per researcher in late July - my best guess is that's when internal employees gained access to the model later released as GPT-6 Astra.

# 11:57 pm / ai, openai, generative-ai, chatgpt, llms, coding-agents, november-2025-inflection, recursive-self-improvement

The purpose of DNS is to spread scams. Terence Eden shares some daunting statistics in support of his take that "the Domain Name System's purpose seems to be a vector for criminals to run scams on people at a terrifyingly high rate".

On this Interisle report (via Andrew Campling), Terence says:

It says 85 million new registrations of gTLDs were made in 2025. Of those 8.5 million were added to blocklists by May 2025. It reckons that a 10% abuse rate is the likely floor for these numbers and it's probably closer to 20%. One in five newly registered domains with a gTLD are scams. That's a bloody crisis.

I had no idea. Apparently ICANN have been discussing this problem for years.

# 2:40 pm / dns, scams, terence-eden

Introducing GPT-6 Astra for developers (via) Blink and you'll miss it, but there's a familiar creature at 1m59s:

Across the board, Astra has more attention to detail, better understanding of the user's prompt, and can build more sophisticated outputs. In particular, it excels at building 3D models. I've seen it make incredible renderings of gardens, shipyards, animals, cityscapes, even Dyson spheres.

Astra really does believe in putting a red neckerchief on a pelican riding a bicycle.

# 11:27 pm / ai, openai, generative-ai, llms, pelican-riding-a-bicycle, gpt-6-astra

GPT‑6 Astra (via) GPT-6 Astra is "rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and AWS" - I've not tried it yet myself, so I don't have a great deal to say about it yet.

It's going to be API priced at the same rate as Claude Fable 5 and 5.1: $10/million input and $50/million output. This is clearly OpenAI's Fable competitor, and appears to score higher than Fable on most of OpenAI's self-reported benchmarks.

Most impressively, Astra scores 99.9% on the recent (released in March) ARC-AGI 3 benchmark - though notably Fable 5 does not yet have a published result, and the ARC-AGI blog notes that the 99.9% score was achieved for $19K using OpenAI's custom "Provider Adapter harness", while the default ARC-AGI harness scored 62.7% for $26K.

The Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work.

Unsurprisingly, given the recent Hugging Face incident, Astra is a beast at security tasks. It scores 100% on ExploitBench (GPT-5.6 Sol got 78.5%), 42.4% on ExploitGym (Sol got 30.3%), and 99.2% within four attempts on SRE-Bench binary reverse engineering compared to Sol's 68.7%.

It's also better at long context: on OpenAI's eight-needle benchmark it got 100% at 256K–512K tokens and 96.3% at 512K–1M tokens. OpenAI may have vanquished one of the ongoing challenges with long context processing.

It doesn't win at everything though. Artificial Analysis note that Astra is still beaten by Fable on their Intelligence Index:

Sits beside GPT-5.6 Sol in Intelligence: GPT-6 Astra scores equal to GPT-5.6 Sol in the Index at 61. This is 5 points lower than Claude Fable 5.1 (max with fallback). The model also trails Meta’s newly released Muse Spark 1.3 (max).

It did better on their Coding Agent Index:

Leads Coding Agent Index cost efficiency frontier: At max effort, GPT-6 Astra costs about the same as GPT-5.6 Sol (max) while scoring 2 points higher on the Index. Per task, the model is less than half the cost of Claude Fable 5, for the same score.

I'll write more about Astra once I get access to it. The API model label once it rolls out will be gpt-6-astra.

# 8:18 pm / ai, openai, generative-ai, llms, llm-release, gpt-6-astra

Python 3.15.0 candidate 2 is here! (via) Hugo van Kemenade (release manager for Python 3.14 and 3.15) announces the final release candidate for Python 3.15, scheduled for release in October:

Entering the release candidate phase, only reviewed code changes which are clear bug fixes are allowed between this release candidate and the final release. [...]

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.15 during this phase, and publish Python 3.15 wheels on PyPI to be ready for the final release of 3.15.0, and to help other projects do their own testing. Any binary wheels built against Python 3.15.0 release candidates will work with future versions of Python 3.15.

Back in 2021 I found a bug in Python 3.10 by running my test suites against it... but I hadn't done this during the RC period, so that bug had already shipped! Since then I've always paid much closer attention to these RCs.

The new RC isn't available for GitHub Actions just yet - keep an eye on actions/python-versions for that. For the moment though you can add this to a testing matrix:

strategy:
  matrix:
    python-version: ["3.14", "3.15"]

steps:
  - uses: actions/setup-python@v7
    with:
      python-version: ${{ matrix.python-version }}
      allow-prereleases: true
      check-latest: true

The allow-prereleases and check-latest flags mean that today this will test against RC1, and when RC2 lands it will automatically switch to that version (and then the stable version once that comes out.)

Update: Datasette passes, sqlite-utils passes, LLM is currently blocked waiting for a 3.15 wheel for scikit-learn, which is optionally used in the test suite.

# 2:59 pm / open-source, python, github-actions

Introducing wrapture. New from Graham Dumpleton (of wrapt, mod_wsgi, and New Relic's Python agent fame), who describes Wrapture as taking the monkeypatching ideas from wrapt and extending them to apply to testing and tracing at the same time.

Wrapture (full documentation here) makes it easy to wrap any function or method such that all access can be traced, or can be overridden to return a different value.

It acts as both an alternative to unittest.mock and a way to implement tracing against an existing project:

Attaching observation to code you do not control, recording what flows through it, and doing so without disturbing the program being watched, is a problem I have never really stopped thinking about.

Wrapture includes OpenTelemetry support and even has an entirely configuration-based mechanism for adding tracing to an existing Python project, which looks like this:

capture = "summary"

[[observe]]
target = "domain:Calculator"
name = ["outer", "inner"]

[[sink]]
type = "jsonlines"
path = "trace.jsonl"

This is still a very young project - just a few weeks old - but it's off to a very promising start.

Interestingly, this is also Graham's first attempt at large entirely agent-driven project:

Every line of code and documentation in wrapture was written by an AI assistant working under my direction. I want to be upfront about that, and equally upfront about what it was not. This was not vibe coding, where a one-shot prompt produces a pile of generated code and the person driving hopes for the best because they lack the knowledge to judge what came back. Vibe coding has earned its bad reputation. I engineered wrapture carefully from the start. I have spent a long time in this particular corner of Python and knew exactly what the result needed to be, and the AI was the means of producing it rather than the source of the design.

In a follow-up post, Unit testing with wrapture, Graham shows the testing patterns supported by the new library:

def test_stub_with_wrapture():
    with wrapture.binding(
        Gateway, "charge"
    ).on_call.returns({
        "id": "stub", "amount": 0}
    ):
        assert OrderService().place(
            500
        )["id"] == "stub"

And this neat example of a test that calls and then modifies the return value from the original method:

def test_pinned_result_with_wrapture():
    charge = wrapture.binding(
        Gateway, "charge"
    )
    charge.on_call.transforms_result(
        lambda r: {**r, "id": "ch_TEST"}
    )
    with charge:
        assert OrderService().place(
           500
        ) == {
            "id": "ch_TEST", "amount": 500
        }

(In both of these examples the OrderService().place(...) method calls Gateway().charge(...).)

# 11:59 pm / graham-dumpleton, monkey-patching, python, testing, pytest, observability, ai-assisted-programming, agentic-engineering, opentelemetry

Introducing Hy4 Preview. New open weight text input (no vision) LLM from Chinese company Tencent today: 770B total parameters, 49B active parameters, 1M token context window, 1.56TB on Hugging Face.

This is a big size increase from their previous Hy3 in July, which was 295B, 21B active, 256,000 context, 598GB.

I recently started using model chat templates to better understand their capabilities. Here's Hy4's chat_template.jinja on Hugging Face, which includes this section:

{%- if not reasoning_effort is defined %}
    {%- set reasoning_effort = 'high' %}
{%- elif reasoning_effort not in ['high', 'no_think'] %}
    {%- if reasoning_effort is none %}
        {{- raise_exception('reasoning_effort error : None, should be no_think/high') }}
    {%- else %}
        {{- raise_exception('reasoning_effort error : ' + reasoning_effort + ', should be no_think/high') }}
    {%- endif %}
{%- endif %}

So it looks like there are just two reasoning effort levels: "high" (the default) and "no_think" (reason by disabled).

I tried my "Generate an SVG of a pelican riding a bicycle" prompt with the default high reasoning via OpenRouter and got this:

Quoting the reasoning trace:

[...] Let's maybe add a helmet? It could improve riding theme, but may obscure head. Maybe a small cycling cap or helmet? The user didn't ask; can add red helmet? Might be cute. But pelican with big beak; a helmet might obscure. Better maybe no.

Maybe add sunglasses? no.

Maybe add water? no.

It's interesting how the reasoning trace uses slightly truncated English, presumably because perfect grammar isn't useful or token efficient for hidden reasoning text.

# 11:53 pm / ai, generative-ai, llms, pelican-riding-a-bicycle, llm-reasoning, llm-release, ai-in-china