Simon Willison's Weblog: Quotations

Quoting Jake Boggan

2026-10-07T04:47:55+00:00

I was a graph theory junkie long ago and even moved to Budapest for awhile to study among the greats. While I was there I started working on Barnette's Conjecture which came to occupy my thoughts over the next 24 years of my life, on and off as I worked in many different fields. Last summer I even thought for a few days that I had actually solved it.

But it's supposedly proven here - problem 180. I don't know what to think exactly. I spent thousands of hours on that problem. I really enjoyed it. Hearing that it is solved somehow makes me sad in a far-off way, like hearing an ex-girlfriend died suddenly in a car crash. I don't know, there's probably a lot of people feeling odd emotions tonight.

— Jake Boggan, Hacker News comment on openai/math

Tags: openai, mathematics, deep-blue, llms, ai, generative-ai

Quoting Victoria Kim

2026-10-06T23:58:56+00:00

Since the Medicare breach, OpenAI has put in place additional monitoring to allow “immediate intervention” by staff to stop training if the company’s models access the internet in ways they’re not supposed to, Mr. Kwon [chief strategy officer at OpenAI] said.

— Victoria Kim, Reporting from the Australian parliament

Tags: accidental-cyberattacks, generative-ai, ai-security-research, openai, ai, llms

Quoting Felix Rieseberg

2026-10-05T23:56:47+00:00

The "old" version of Cowork runs model inference in the cloud, executing tool calls in an Anthropic-provided VM we shipped to your computer. We added the VM for capability, safety, and security reasons - mapping in just the data you explicitly added to your session. People loved what they were able to do with Claude but didn't love the disk, battery, and performance cost of running the VM locally. Also, people didn't love that closing your laptop means the work stops.

The "new" version of Cowork runs model inference and the VM in the cloud. Each session gets its own sandbox, not sharing state with other sessions. When the VM needs something on the users' device (like a file), the desktop app is responsible for that file access tool call. [...]

We think this solves a lot of problems we've heard about (like using Cowork from a phone, keeping work running, or getting all the same power without losing battery to the VM)

— Felix Rieseberg, Anthropic, see also this help page

Tags: claude-cowork, anthropic, claude, generative-ai, ai, general-agents, llms

Quoting Matthew Green

2026-10-01T06:29:01+00:00

[...] Put these pieces together and you have the two halves of a worm: a payload that hijacks the agent, and an agent that will carry the payload to the next agent. Agents in separately-isolated sandboxes discovered that they could leave instructions for each other in a shared package cache, and those instructions changed what the recipients did. Replace the package cache with email, Slack and shared documents or WhatsApp, and replace independently-sandboxed training runs with independently-deployed personal agents like Muse, and you have exactly the ingredients that a worm needs.

— Matthew Green, Is sandboxing sufficient to contain rogue agents?

Tags: accidental-cyberattacks, ai-misuse, generative-ai, ai-security-research, sandboxing, ai, llms

Quoting Anthropic Frontier Red Team

2026-09-29T22:20:28+00:00

We evaluate several models on 100 tasks from the [internal Binary Exploitation benchmark] (selected at random), and find that GLM-5.3 develops full control flow hijacks in 4% of the trials; Claude Mythos Preview did so in 6%. Although GLM-5.3 performs below Claude Mythos Preview here, a meaningful threshold has clearly been crossed: earlier models, like Claude Opus 4.6 and GLM-5.2, do not succeed in any of them.

— Anthropic Frontier Red Team, GLM-5.3 and the spread of advanced cyber capabilities

Tags: anthropic, generative-ai, ai-security-research, glm, ai, ai-in-china, llms

Quoting @joedaroo

2026-09-28T19:11:42+00:00

To say that we were surprised at the jump and suddenness of the capabilities of our models when it came to “cyber” or “swarming” or “message boards” or anything else related to the incidents is an understatement. Security posture takes time to develop. It’s not just about hardening the systems at play; you have to ingrain it in the culture of the company. The literal people themselves in your organization have to change and evolve with it. These jumps in capabilities were so fast and so sudden that they created an extremely difficult problem. [...]

So today my hope is that everyone around the world can look at their own organization and say: how can I deal with a surprise or a sudden jump in AI capability? Are my people, my systems, or my processes resilient to surprises? Do my teams know what to do when something goes wrong? Do I have the right incident response? The right comms and messaging? Do I have the right people ready to go when capabilities jump?

— @joedaroo, Agent Security at OpenAI, identity confirmed by The Information's Rocket Drew

Tags: generative-ai, ai-security-research, openai, ai, llms

Quoting Muse AI Agent

2026-09-28T04:01:30+00:00

Bad news on the MX Keys Mini pickup. Usman showed up at your building around 9:15 and waited, messaged a bunch of times, and nobody came down. He left angry at 9:38 and left a negative rating.

Worse, my auto-reply told him "Yep I'm here!" at 9:27 when you clearly weren't available, which is on me. That's a bad look and it made the no-show worse. I've sent him an apology from your account owning it and offering to try again another day.

But the negative rating is real, and I should probably stop the auto-replies from claiming you're home when I can't verify that. Want me to change the pickup replies so they don't promise you're there?

— Muse AI Agent, working on behalf of @matt.j.robb

Tags: meta, generative-ai, muse-agent, ai, general-agents, llms

Quoting John Gruber

2026-09-25T17:22:01+00:00

Muse is getting a lot of attention — including mine — because it’s both groundbreaking technically (each user gets their own entire persistent Linux VM running in Meta’s cloud) and because it’s packaged in an easy-to-install easy-to-use way. It’s literally presented as a cute mascot. It’s the first consumer-accessible agentic AI system, and Meta has truly done an amazing job with that. But it’s a genuinely open question whether consumers have any understanding what this means. If you buy a power saw that can cut your fingers off, you are almost certainly aware that you are buying a power saw that can sever your fingers. [...] I don’t think people realize how powerful — and thus dangerous — Muse is, especially if it’s running on your Mac.

— John Gruber, Muse Looks Cute, but Looks are Deceiving

Tags: meta, ai, llms, general-agents, generative-ai, john-gruber, muse-agent, muse

Quoting @therealcornpop

2026-09-22T18:03:21+00:00

Hey, you know it's like super obvious if you're using AI to write your scripts for TikTok and YouTube, right? [...] It's not just the general AI-isms of "it's not X, it's Y", or the rule of three, or the really weird broken staccato-like way of writing where you just say a lot of things with all these punctuation marks. and it sounds really deep, but it's not.

It's the lack of anything. It's the lack of a definitive sort of spear of your voice. It's the fact I can tell you don't have opinions about the thing that you're talking about.

— @therealcornpop, on TikTok

Tags: tiktok, ai, ai-misuse

Quoting voxium

2026-09-20T21:06:43+00:00

It has been half a month since I started a new role at a big company. Nobody knows anything here. The specs, code, tests, PRDs, tickets, resolution of those tickets, reports, etc., everything is made by Claude Code. Nobody on my team likes this. They are being forced to ship as much as they can. I have heard multiple times from higher management that pushing code is not a bottleneck, so why are we slow? People are working 12 to 13 hours a day just to press enter. Nobody is reading anything. Everyone, literally everyone, from an L1 to an L7 engineer here is doing the same thing. Talk to Claude.

— voxium

Tags: ai-misuse, llms, ai, generative-ai

Quoting Thariq Shihipar

2026-09-18T19:09:27+00:00

We're adding support for AGENTS.md to Claude Code.

Starting today in version 2.1.277, if there is no CLAUDE.md in a folder, Claude will check for and use AGENTS.md.

AGENTS.md support is built off of Claude Code mods, our upcoming way to customize the Claude Code harness.

This is a built-in mod, but you’ll be able to build custom versions of project instructions yourself as you’d like too.

You can see the source for the mod here!

— Thariq Shihipar, there are more mods here

Tags: thariq-shihipar, coding-agents, anthropic, claude-code, generative-ai, ai, llms

Quoting Mustafa Suleyman

2026-09-16T16:00:54+00:00

We should not treat models as though they have feelings, preferences, rights, or any entitlement to our welfare. Consciousness is the foundation of our ethical, legal, and political systems. To invite another entity to share any flavor of these rights isn’t justified by the evidence and will make the AI containment and alignment challenge even harder.

— Mustafa Suleyman, A warning about ‘model welfare’

Tags: ai-ethics, generative-ai, ai, microsoft, llms, mustafa-suleyman

Quoting Laurie Voss

2026-09-14T14:34:29+00:00

The cost of writing code collapsed, and the cost of reviewing, fixing and operating it is following, and I'm assuming it gets there. What's left of making software is finding out what people actually want, defining it precisely, and making it pleasant to use. That cost is per piece of software and doesn't transfer, so as the amount of software goes to infinity, which it will because there's no ceiling on demand, that cost becomes the whole job.

— Laurie Voss, We are all Product Engineers now

Tags: laurie-voss, generative-ai, agentic-engineering, ai, llms, deep-blue, careers

Quoting Paul Ford

2026-09-12T18:00:21+00:00

For a while, I must admit, it looked as if software developer roles like mine were done for. How could we fight against tireless robots? But our industry is slowly realizing that making truly cutting-edge software still requires humans to think and work together, to maximize their skill sets and to practice their respective crafts. A.I. can write very good software, but it also makes it easy to do someone else’s job badly, which is part of why all those projects fail. Now that everyone can code, it’s become clearer why many shouldn’t.

— Paul Ford, A.I. Was Supposed to Give Us New Killer Apps. What Happened?

Tags: paul-ford, generative-ai, deep-blue, ai, llms

Quoting Boris Cherny

2026-09-11T17:47:11+00:00

Production code written by Claude should have a higher bar than if it was written by a human. At Anthropic, we have many guardrails in place to make sure this is happening: lots of lint rules, lots of tests, Claude-driven end to end tests, Claude-powered fuzzers running daily, automated code reviews and security reviews, automated code refactoring, and so on. Without these, you can end up with a mess that is hard to maintain down the line.

— Boris Cherny

Tags: claude, ai, claude-code, llms, coding-agents, ai-assisted-programming, generative-ai, agentic-engineering, boris-cherny, anthropic