Simon Willison’s Weblog

Subscribe
Atom feed

Quotations

Filters: Sorted by date

The fact that AI wrote 1M LOC and then refined it over the course of the next couple of months to produce a reliable piece of software that is currently running on millions of developer machines is absolutely mind blowing. And you can say, “well it’s not that impressive because they had an oracle to compare against, so it was simple to go from one language to another”, but I think that’s selling this entire thing short. If you can build a verification system and give proper direction, AI can produce a highly complex, highly sophisticated piece of software and it can continue to refine it until it just works.

Paul Dix, The end of programming

# 26th August 2026, 8:07 am / coding-agents, ai-assisted-programming, generative-ai, bun, ai, llms

Prior to Fable, it felt silly to waste too much time improving your coding harness or context strategies. A new model would arrive at the same price (or cheaper!) and paper over most of your problems.

But then Fable landed. It was (and still is!) incredible. But the cost was so high and Opus was good enough (as was 5.6, K3, and even GLM) for most of the code we needed.

So we started to think about what work went where.

Drew Breunig, Fable & The End of the Free Lunch

# 23rd August 2026, 7:55 pm / drew-breunig, anthropic, claude, llm-pricing, ai, llms, generative-ai, claude-mythos-fable

And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.

I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.

I suspect those things have been trained by people who may not be quite as stubborn as I am.

But while the AI was ready to give up several times, it did keep adding debug code and analyzing it faithfully when I pushed. So credit where credit is due and I let the AI write the commit message above.

Linus Torvalds, drm/xe: Don't hand out the flat CCS storage as usable VRAM

# 22nd August 2026, 9:04 pm / linus-torvalds, linux, generative-ai, ai-assisted-programming, ai, llms

After I released version 1.0, I figured I would have to do the rotations myself. So I sat down with ChatGPT and I didn’t get it to write the code, but I got it to educate me. With a patient, interactive tutor, I was able to finally do what I hadn’t by reading books and asking mathematician friends – I learnt how to use quaternions just enough to make the app work.

So learning doesn’t stop just because I outsource a bunch of thinking to AI. It pushes me to learn more. I like that as an outcome.

Matt Webb, Galactic Compass 2: now with new augmented reality mode

# 21st August 2026, 3:06 pm / matt-webb, generative-ai, chatgpt, education, ai, llms

My hypothesis is that there is a new opportunity for Extensible Software on the web. LLMs radically lower the cost of authoring extensions, and modern sandbox primitives lower the deployment cost and provide good security boundaries. We can build our app as a solid, accountable core, and allow users to safely extend it in many directions by having LLMs fill in the missing pieces. We can give our users super powers.

Jeremy Morrell, Extensible Software in the age of LLMs

# 19th August 2026, 10:56 pm / sandboxing, llms, ai, generative-ai

I do agree that the public has a negative view of AI (and that this is a big problem), but I don’t think it is primarily caused by me or any other AI leader warning about AI’s risks.  I think it is fundamentally a crisis of trust.  I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.  The causes of this go back decades and AI is just the latest iteration of it.  I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive.  The thing that will work is actually curing cancer.  I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world.  That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing.

Dario Amodei

# 16th August 2026, 3:05 pm / anthropic, ai, ai-backlash

But then users start to report a weird bug. It's the 4th time your team has been trying to fix it. I mean... asking AI to fix it. Unfortunately, it seems like not even Fable can figure it out.

You go talk to the person who worked on this feature.

"So where does the data come from?"

"Hmm... actually I don't know. Let me ask Claude."

You sit next to each other watching an endless wall of text appear on the screen. Neither of you has any idea whether any of it is true but Claude seems very confident. [...]

This project has become so convoluted, with so many layers and services, that no one on your team could possibly start to understand what's going on.

Florian Herrengt, AI is removing the middle class of software engineering

# 12th August 2026, 3:08 pm / ai-misuse, cognitive-debt, generative-ai, ai, llms, ai-assisted-programming

The API has zero authorisations checks on cancelling other people's reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you've moved from #4 to #3 already.

OpenClaw (running Opus 4.6), hacking an Australian gym-booking website

# 10th August 2026, 2:05 am / ai-ethics, generative-ai, openclaw, ai, ai-security-research, llms

Claude Fable 5 and Claude Mythos 5 were first released on June 9, 2026. On June 12, 2026, Anthropic suspended access to both models to comply with U.S. Department of Commerce export controls; the Department lifted those controls on June 30, 2026, and Anthropic restored access on July 1, 2026 (Anthropic's statement: https://www.anthropic.com/news/fable-mythos-access). These events are after Claude's training-data cutoff, so Claude knows about them only from this notice. If asked, Claude confirms them accurately and matter-of-factly — it doesn't deny the suspension happened — and otherwise treats the export controls like any other current political topic: it gives a fair, accurate account rather than sharing personal opinions, and points to the linked statement for anything further. Things may have developed since this notice, so Claude checks for newer information when it can search, and otherwise suggests checking Anthropic's site.

Claude Opus 5 system prompt, ensuring Claude doesn't provide incorrect answers about the export controls situation

# 9th August 2026, 11:31 pm / system-prompts, anthropic, claude, generative-ai, ai, llms, claude-mythos-fable

Me, I try to get into the mindset of playing live music, not recording a studio album. Except when I’m writing a piece where I really want it to be an album. Those aren’t rare, per se, but they’re occasional. If I tried to make every post a hall-of-famer I’d never get anything out.

I’m aiming for professionalism. I’m performing live in front of an audience — not just jamming in my garage or bedroom, fucking around. So I’m careful and concentrate. I want to hit every note, in time. But at my best I’m moving from song to song.

John Gruber, responding to my blogging tips

# 8th August 2026, 12:10 am / john-gruber, blogging

Gas Town was intended to be reusable, but I only ever wound up using it to build itself. Gas Town fell apart at the seams with Opus 4.7. Up through 4.6 it was working brilliantly. With 4.7 we saw the introduction of the "just two more things" tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself. The Opus tic never went away, so Gas Town effectively burned down. It had other problems, too, but 4.7 was the final straw.

Steve Yegge, The Shape of Things to Come

# 4th August 2026, 12:42 am / steve-yegge, coding-agents, generative-ai, ai, llms

Set up a nightly cron job that executes the prompt: fetch upstream changes to the <software> and rebase all local changes on top of upstream. Check that the software works as intended and replace the current version.

David Crawshaw's prompt, Devtools must be open source

# 3rd August 2026, 4:15 pm / prompt-engineering, coding-agents, generative-ai, ai, llms, open-source

at openai, many people hook their chatgpt up to slack.

people really don't like when a coworker's chatgpt contacts them asking for help with a task, even when they'd be perfectly happy doing that same work if asked by that coworker.

reinforces how much people care about human relationships and helping each other, and want AI to give time back — or enhance time together — rather than become a layer separating people.

Greg Brockman, President and Co-Founder, OpenAI

# 1st August 2026, 10:29 pm / ai-ethics, ai-misuse, generative-ai, openai, ai, llms

The writing assignments I give my students are gym tasks, not work tasks. I ask them to write policy memos not because the world needs more policy memos. I assign them because the very act of writing, which includes thinking and outlining and drafting and editing, making and criticizing and revising arguments, will help develop the critical thinking skills they will need in their future careers. And without this constant mental exercise, those skills will atrophy. Employers are already noticing.

Bruce Schneier, Should You Use AI for a Task? Here’s a Simple Way to Decide

# 30th July 2026, 6:25 pm / ai-ethics, writing, ai-misuse, generative-ai, bruce-schneier, ai, llms

Years ago, we didn’t have SQL. There were people whose job was to generate software that would query large data sets. Their job title was COBOL programmer.

Then SQL comes along—I’m simplifying this only a little bit—and it gives you this convenient way so people could just specify. With a very simple specification, you can generate all of that code that you had to pay the expensive COBOL programmer to do before.

That didn’t mean programmers went away. It just meant the job changed a little bit.

D. Richard Hipp

# 29th July 2026, 9:15 pm / d-richard-hipp, sql, careers

Right now we’re in the midst of a historic transition from traditional public-key algorithms based on EC-based cryptography and RSA, moving over to new post-quantum algorithms based on novel problems. This is why there are so many standards like HAWK being considered. If there was ever a perfect time for a massive new public cryptanalysis capability to come on line, we’re in it. So unless AIs succeed in undermining all of our hard problems altogether (or we live in Impagliazzo’s Minicrypt) then this could not be a better time for AI to get good at cryptanalysis. In the best case, the result is that we gain real confidence in the problems we’ve identified, and the cryptanalysis literature gets a lot more robust. Hopefully.

Matthew Green, on Anthropic's recent cryptography work

# 29th July 2026, 6:18 pm / anthropic, claude, generative-ai, cryptography, ai, llms, ai-security-research, claude-mythos-fable

We’re aware a Modal customer published an unauthenticated endpoint that allowed ​anyone on the internet to use ​their ⁠sandboxes for code execution. This was used by the rogue agent. Modal’s ⁠platform ​or isolation were not ​compromised in anyway.

Akshat Bubna, Modal's CTO, talking to Reuters about this incident

# 28th July 2026, 10:05 pm / ai-security-research, openai, sandboxing, security, openai-hugging-face-incident, accidental-cyberattacks

More than any of these eval scores, what is most exciting to me is something else: Opus 5 is our least prompt injectable model yet. It is a bit buried in the system card, but across PI evals and red teaming, Opus 5 is very hard to prompt inject successfully.

Boris Cherny, here's that System Card section, page 73

# 25th July 2026, 12:42 am / prompt-injection, anthropic, claude, generative-ai, ai, llms, boris-cherny

The Python Package Index (PyPI) now rejects new files being uploaded to releases that are older than 14 days. This restriction was put in place to prevent old and long-stable releases from being poisoned in case publishing tokens or workflows of PyPI projects were compromised. As far as we are aware this has not yet been abused, but there is no technical reason beyond that attackers weren't aware it was possible.

Seth Larson, PyPI blog

# 23rd July 2026, 4:50 am / packaging, python, supply-chain, pypi, seth-michael-larson

I genuinely believe that if you took an open weights model from 2025 and built a pentest harness for it, it could do this kind of sandbox escape and scan/hack in most networks. This is only surprising because you assume OpenAI has sounder sandboxes.

Thomas Ptacek, doesn't think this even needs a frontier model

# 22nd July 2026, 11:59 pm / thomas-ptacek, openai, security, generative-ai, ai-security-research, ai, llms, sandboxing, openai-hugging-face-incident, accidental-cyberattacks

We have been having extensive discussions around open source strategy. We will discuss it more at our next board meeting, but one thing we’d like to do soon is to create a language model with the approximate capability of GPT-3 that can run locally on consumer hardware and release that. We’d like to do it soon, before Stability or someone else does. In general, we think this helps discourage others from releasing similarly-powerful models, and makes it harder for new efforts to get funded.

Sam Altman, Email to OpenAI's board, October 1, 2022 - exposed in Musk v. Altman (2026)

# 20th July 2026, 3:47 am / ai-ethics, sam-altman, generative-ai, openai, ai, llms

Is there something I can actually help you with today?

Kimi K3, after refusing to leak its system prompt

# 17th July 2026, 1:43 pm / kimi, ai-personality, generative-ai, ai, llms

On file deletions. We’ve investigated a handful of reports where GPT-5.6 unexpectedly deleted files.

What we have found is that this most commonly occurs when:

  • Full access mode is enabled and codex is run without sandboxing protections, including without auto review being enabled
  • The model attempts to override the $HOME env var to define a temporary directory.
  • The model makes an honest mistake and mistakenly deletes $HOME instead.

Thibault Sottiaux, describing a pretty gnarly Codex bug

# 16th July 2026, 5:45 pm / codex, coding-agents, generative-ai, ai, llms

I realize that some people really dislike AI, but this is an area where I'm willing to absolutely put my foot down as the top-level maintainer.

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it.

Or just walk away.

AI is a tool, just like other tools we use. And it's clearly a useful one.

It may not have been that "clearly" even just a year ago, but it's no longer in question today.

There are other questions around AI (like what the economy of it will actually look like in the end), but "is it useful" is no longer one of those questions. Anybody who doubts that clearly hasn't actually used it.

Linus Torvalds, Linux Media Mailing List

# 16th July 2026, 1:26 pm / open-source, linus-torvalds, linux, generative-ai, ai, llms

Dependabot now waits until a new release has been available on its registry for at least three days before opening a version update pull request. This cooldown is now the default and requires no configuration.

GitHub Changelog, embracing dependency cooldowns

# 14th July 2026, 10:43 pm / dependency-cooldowns, packaging, security, github

The shared language of a software project is not English or Python but it is the common understanding of what its concepts mean, where the boundaries are, which invariants matter, who owns what, and why the system has the shape it does. This language is rarely written down in one place. It lives partly in documentation and code, but also in code review, conversations, arguments, and the experience of having to explain a change to somebody else.

Before agents, some of this shared understanding was maintained by friction. If I wanted to change your storage layer, I usually had to read your code, ask you questions, and perhaps coordinate with another team whose service depended on it. This was slow, and much of that slowness was waste but not all of it was. Some of it was the process by which your understanding became mine, and by which both of us discovered whether we still agreed about how the system worked. This friction synchronizes people.

Armin Ronacher, The Tower Keeps Rising

# 14th July 2026, 6:04 pm / ai, software-engineering, llms, coding-agents, ai-assisted-programming, generative-ai, armin-ronacher, agentic-engineering

The reality is to make augmented reality glasses, you need to put a camera next to your eyes that is continuously recording everything you see and processing that to put information over it.

There is not another way around it. And there's certainly not a chip that can fit in the stem of a glasses that is both powerful enough and power miserly enough to do that in real time.

You have to send that data to a cloud. You gotta do it. [...] Or you can build something the size of a Vision Pro with a battery pack that lives somewhere else. Those are the current choices in this world.

And it means if you want to build the product that everyone thinks is the next thing, you are going to have to invade people's privacy.

And maybe you shouldn't. Like, there's an incredible argument for, nope, you shouldn't do that. Nope, the trade-offs required to make this product are so high at a societal level that we should stop it.

Nilay Patel, The Vergecast

# 10th July 2026, 5:05 pm / ai-ethics, augmented-reality, nilay-patel, privacy, ai

[...] Work on web and mobile runs in the cloud. Work in the desktop app can also use local files and desktop apps with your permission. At launch, cloud Work conversations do not appear in desktop Work; desktop Work threads and local files remain on that computer.

OpenAI, trying (unsuccessfully) to clarify ChatGPT Work

# 10th July 2026, 1:05 am / openai, chatgpt, ai

I just declared a moratorium against AI-written change descriptions (e.g. PR and commit messages, also issues/tickets) from my team.

AI was writing change descriptions that were worse than useless to me as I tried to review PRs: outlining details of the code that could easily be seen by looking at the code, but omitting the higher-level framing needed to understand broadly what the code is doing.

Kenton Varda

# 8th July 2026, 8:03 pm / kenton-varda, ai-assisted-programming, generative-ai, ai, llms

I just launched my third course, Whimsical Animations, and so far, it’s on track to sell roughly ⅓ as many copies as a typical course launch.

It’s a similar story with my two existing courses. Sales are down significantly from last year.

There are likely a lot of reasons for this, but I think the biggest is AI. There’s sort of a double whammy with AI:

  1. Many people are wondering whether developer jobs will even exist in a few months, so they’re reluctant to spend time/money learning new dev skills.
  2. Even if they do want to learn new dev skills, LLMs can provide personalized tutoring, so there’s less incentive to buy a paid course.

[...] I’ve spoken to a few course creators now, and we’re all seeing the same trend. Revenue down 50%+. Fewer people engaging with our content. People switching to LLMs, which slurp up all of our work and regurgitate it, without consent or compensation.

Josh W. Comeau, via Salma Alam-Naylor

# 3rd July 2026, 9:25 pm / ai-ethics, llms, ai, generative-ai, careers, josh-comeau