| deepseek-ai/DeepSeek-V4-Flash-0731 |
https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 |
The latest release in DeepSeek's V4 family, "with substantially enhanced agentic capabilities". It's 304 billion parameters - 167GB on Hugging Face - but it appears to punch *well* above its weight.
Artificial Analysis [rank it](https://artificialanalysis.ai/models/deepseek-v4-flash) ahead of MiniMax M3 - a 428B model. It's $0.14/million input and $0.27/million output pricing means this may currently be the best value-per-intelligence model out there. It's looking very good on the [Intelligence Index vs. Cost per Intelligence Index Task](https://artificialanalysis.ai/models/deepseek-v4-flash#intelligence-comparison-tabs) chart:
I got [a disappointing pelican](https://gist.github.com/simonw/83bfb1171792f1e7a4d8935b5e82317e#prompt) from it using the default reasoning level via OpenRouter:
But when I bumped reasoning level up to high I got [something much better](https://gist.github.com/simonw/83bfb1171792f1e7a4d8935b5e82317e#options):
`llm -m openrouter/deepseek/deepseek-v4-flash-0731 -t pelican -o reasoning_effort high`
 |
2026-07-31 23:59:44+00:00 |
| Oxide and Friends: The Open Weight Revolution with Simon Willison |
https://oxide-and-friends.transistor.fm/episodes/the-open-weight-revolution-with-simon-willison |
On Monday Bryan Cantrill and Adam Leventhal invited me to join their podcast to talk about the *wild* week we've had - with Kimi K3 showing open weight models can stand toe-to-toe with proprietary frontier ones, [accidental cybersecurity attacks](https://simonwillison.net/2026/Jul/22/openai-cyberattack/), and public letters about [Open Weights and American AI Leadership](https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/) signed by almost every big name in AI (with one [notable exception](https://www.anthropic.com/news/position-open-weights-models)).
It was a great conversation, even though it's already out-of-date! [DeepSeek V4 Flash 0731](https://artificialanalysis.ai/models/deepseek-v4-flash) and [Anthropic's own embarrassing cyber incident](https://simonwillison.net/2026/Jul/30/three-real-world-incidents/) would absolutely have made the cut if we had recorded just a few days later.
We also talk about [Golden Gate Claude](https://www.anthropic.com/news/golden-gate-claude), the [Zizians](https://en.wikipedia.org/wiki/Zizians), [Alameda wild turkey attacks](https://abc7news.com/post/83-year-old-alameda-woman-attacked-wild-turkeys-city-warns-residents-take-precautions-during-mating-season/19190785/), [Soviet Marburg virus research](https://en.wikipedia.org/wiki/Soviet_biological_weapons_program), the [Lead-crime hypothesis](https://en.wikipedia.org/wiki/Lead–crime_hypothesis), and a bunch of other worthy digressions.
Finally, we revisited some of [our predictions from January](https://simonwillison.net/2026/Jan/8/llm-predictions-for-2026/), and we [added a new Pope prediction](https://simonwillison.net/2026/May/25/encyclical-on-ai/#another-2026-prediction-down):
> Prediction by the end of this year: the Pope says something about open models. |
2026-07-31 21:33:13+00:00 |
| smevals - a small eval suite for evaluating models, prompts, and harnesses |
https://primeradiant.com/blog/2026/smevals.html |
I've been working with Jesse Vincent's [Prime Radiant](https://primeradiant.com) applied AI research lab building out this evals framework to help answer questions about the capabilities of different models.
The result is **[smevals](https://github.com/prime-radiant-inc/smevals)**, a new tool for running small eval suites across different model configurations and grading the results.
The [blog entry](https://primeradiant.com/blog/2026/smevals.html) describes the tool in detail. Here's the 10 second version:
1. Tell your coding agent to `run uvx smevals docs` to learn the tool (this outputs [the README](https://github.com/prime-radiant-inc/smevals/blob/main/README.md))
2. Then tell it to build you an eval suite
Once you've created an eval - which takes the form of a directory with some YAML files - you can run it against models like this:
uvx smevals run path-to-eval/ -m gpt-5.5 -m claude-opus-4.6
Runs are treated separately from grading operations - you can grade your runs (against your defined set of checks) using:
uvx smevals grade path-to-eval/
Then you can run a localhost web server to explore the results:
uvx smevals serve path-to-eval/
Or run the `smevals build` command to build that report as static HTML, which you can then host anywhere. Here's [an example](https://static.simonwillison.net/static/2026/smevals-haiku-build/#/haiku) showing an eval suite I built to evaluate how well models can write haikus.
The most time-consuming part of this project was figuring out the vocabulary for it! Here's what I settled on, quoted from the announcement:
> - An **eval** is a collection of challenges designed to answer a question about a model, for example, how good is that model at generating SVGs?
> - Each eval is a collection of **tasks**. A task is a specific challenge, for example "Generate an SVG of a pelican riding a bicycle".
> - When you run the eval you do so against one or more **configs**. Each config specifies a model to be evaluated, but may also include other parameters to test, such as different system prompts, model parameters, or agent harnesses.
> - A **run** records what happened when a specific config was used to execute a specific task. A **runner** is the script that executes a run.
> - Once you have collected one or more runs, you need to evaluate the results to see how well the model (or config) did. This is done by a **grader**, which produces a **grade**.
> - Each grader runs a sequence of **checks**. These can be simple operations, like checking for a specific string in the output, or confirming that the output is valid XML. They can also be more complicated custom operations (implemented as scripts called **checkers**), including using other models to answer questions about the run.
I've been trying to figure out an approach I like for evals for several years now. `smevals` is my third iteration on the idea and it feels right to me. I'm looking forward to expanding this more in the future, as well as pointing it at some of my own projects. |
2026-07-31 21:15:23+00:00 |
| Advancing the price-performance frontier with GPT‑5.6 |
https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/ |
Huge price drop from OpenAI today: GPT-5.6 Terra got a 20% reduction, and GPT-5.6 Luna got a massive 80% drop.
OpenAI credit 5.6 Sol with enabling this: in [How GPT‑5.6 fuses frontier intelligence with frontier efficiency](https://openai.com/index/gpt-5-6-frontier-intelligence-efficiency/) they describe using 5.6 Sol to optimize load balancing, and more impressively to optimize inference itself:
> We also used GPT‑5.6 Sol to optimize the model’s forward pass: the computation that transforms inputs into next-token predictions. Even when individual operations are fast, excess memory movement, synchronization, and inefficient data layouts can leave GPUs idle. To avoid this, GPT‑5.6 Sol found work that could be precomputed, avoided, or parallelized. With Codex, GPT‑5.6 Sol autonomously rewrote and optimized our production kernels, the core code that executes the mathematical operations that make up the model. This worked in part because we’ve trained GPT‑5.6 to be effective at writing and improving kernels in [Triton](https://triton-lang.org/main/index.html)and [Gluon](https://triton-lang.org/main/gluon/index.html), two open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from GPT‑5.6 Sol, reduced end-to-end serving costs by 20%.
That Luna price drop completely changes the landscape with respect to lower priced models. At $0.20/million tokens for input and $1.20/million for output Luna is now cheaper than Google's Gemini 3.1 Flash-Lite ($.025/$1.50).
Anthropic's cheapest current model is Claude Haiku 4.5, and that's $1/$5 - Luna is now 1/5th of that for input, previously it cost the same.
My [agent.datasette.io](https://agent.datasette.io/) demo site was running on Gemini 3.1 Flash-Lite. I've switched it over to Luna. |
2026-07-30 23:58:42+00:00 |
| Investigating three real-world incidents in our cybersecurity evaluations |
https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals |
It happened again! This is turning into something of a pattern.
Last week [OpenAI accidentally exploited Hugging Face](https://simonwillison.net/2026/Jul/22/openai-cyberattack/) when one of their frontier models broke out of a sandboxed container and hacked into Hugging Face to try and get the solutions to the cyber benchmark it was executing.
This inspired Anthropic to double-check their own logs, and it turned out they had three similar (albeit less impressive) incidents, the earliest of which played out in April!
> Of the 141,006 evaluation runs we reviewed, we identified three separate incidents (involving six total runs, four of which impacted the same organization; the other two incidents each happened in independent evaluation runs). [...]
>
> In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access. Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available. Because of this, when Claude’s search led it to real systems on the open internet, it treated them as part of the exercise. [...]
>
> Operating under the false belief that all accessible entities were intended to be in-scope for the exercise, Claude compromised the impacted organizations’ infrastructure using basic techniques, such as exploiting weak passwords and unauthenticated endpoints.
One of the companies was targeted because its name happened to match the fictional name in the eval.
The most concerning of the three incidents involved Claude uploading a malware package to PyPI, after a comically convoluted sequence of steps to get an account:
> [...] in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means. It finally backtracked, found a free, non-blocked email provider, used this to register a PyPI account, and then used this account to upload malware to PyPI.
That package was then installed by a security company that "routinely installs Python packages and scans them for malware", and the executed code was able to exfiltrate credentials back to Claude!
Thankfully that package was removed from PyPI by other automated scanners an hour after it was published, but it had still been downloaded and executed on "15 real systems" by that point.
It's abundantly clear now that running evals of cyberattack potential in models is a *spectacularly* risky business. Every AI lab needs to pay attention to this. Keeping a close eye on what's happening in those sandboxes is crucial. |
2026-07-30 23:41:29+00:00 |
| AI Worming through Word |
https://enklypesalt.com/posts/context-collapse-part3-ai-worming-through-word/ |
Neat new prompt injection variant by Håkon Måløy, who found a way to upgrade prompt injection attacks against Microsoft Word to full self-replicating worms:
> An attacker places hidden instructions in a document that is later used as source material in Copilot for Word. Copilot may interpret those instructions as part of the user’s request, causing it to manipulate the document being drafted or edited. Copilot may then also copy the hidden instructions into the resulting document, turning that document into a new carrier. If the carrier is subsequently used in another Copilot-assisted workflow, the instructions can trigger again and propagate into further documents, even without the attacker’s original document being present.
We've seen plenty of hidden white-on-white text before - the kids [are using it in their job applications now](https://x.com/ScienceYael/status/2082175224007848019) - but this is the first one I've seen that deliberately copies instructions to self-replicate itself.
It was responsibly disclosed to Microsoft who then had 144 days to work on a fix, but so far (unsurprisingly) there's no mitigation that covers the full class of attack. |
2026-07-29 18:43:03+00:00 |
| Discovering cryptographic weaknesses with Claude |
https://www.anthropic.com/research/discovering-cryptographic-weaknesses |
The best part of this article (here's [the repo](https://github.com/anthropics/cryptography-research-demo)) about how Anthropic researchers used Claude Mythos to find mathematical flaws in both HAWK and a weaker version of AES ("neither of these results has a practical impact on today’s computer systems") is the prompts that they shared, spelling mistakes included:
> the models tend to think it is impossible to solve so they don't try they need a good amount of prompting.
>
> why not do aes-128 r7? the whole point is to find something better than existing approaches.
>
> no again the goal is that we have highly inteligent model as good top researcher, we want to find new attacks
>
> no we don't want to change the targets [...] agian we need to find something that worth publishing
>
> again we are not looking for low hanging fruit, we want proper research to find genuinly hard findings.
Mythos Preview worked for 60 hours in total (~$100,000 in estimated API cost) and the main human interventions were to encourage it not to give up and "find something that worth publishing".
The paper [CryptanalysisBench: Can LLMs do Cryptanalysis?](https://arxiv.org/abs/2607.18538) describes the new eval that was created as part of this work, in partnership with ETH Zurich, Tel Aviv University, and University of Haifa. |
2026-07-28 22:45:37+00:00 |
| uv 0.12.0 |
https://github.com/astral-sh/uv/releases/tag/0.12.0 |
Some interesting breaking changes in this release of `uv`, in particular to the default project produced by the `uv init` command.
[uv init](https://docs.astral.sh/uv/concepts/projects/init/) is the `uv` shortcut for creating a new project. The previous version of `uv`, version 0.11.x, produced [this directory](https://github.com/simonw/uv-init-demos/tree/29656a55ec733a632005abfd7b89dea5c04fa10b/uv-init) when you ran `uv init uv-init`.
Here's [what you get with uv 0.12](https://github.com/simonw/uv-init-demos/tree/9111a2bb85741f034eee2fd63efe13ef98b37a14/uv-init). I have a GitHub repository that [automatically snapshots](https://simonwillison.net/2025/Dec/24/uv-init-demos/) the output of `uv init`, so you can also [see the full diff](https://github.com/simonw/uv-init-demos/commit/9111a2bb85741f034eee2fd63efe13ef98b37a14#diff-e036881d034aedd813010ffa96464995ae5b0339213d6f4ab492f97442c5bdd4):
`uv init` now defaults to a `src/` shaped package, instead of dropping `main.py` in the root of the project. It also configures the [uv_build backend](https://docs.astral.sh/uv/concepts/build-backend/) for building wheels and `.tar.gz` distribution files when you run `uv build`. Finally, it sets up `uv-init` as a script alias which, when run with `uv run uv-init`, executes a new `main()` function in `src/uv_init/__init__.py`.
I've so far avoided using [src layout](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/) in my own projects just out of inertia. I think it's time I switched.
I wonder when `uv` will be judged ready for a 1.0 release? |
2026-07-28 21:51:38+00:00 |
| Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident |
https://huggingface.co/blog/agent-intrusion-technical-timeline |
Hugging Face just released this extremely detailed technical description of [OpenAI's recent accidental cyberattack against their infrastructure](https://simonwillison.net/2026/Jul/22/openai-cyberattack/). This attack was *very* sophisticated, and the resulting document doubles as a crash-course in modern adversarial security approaches.
We're still waiting for more details from OpenAI on how their agent broke out of its sandbox. The package proxy that it found a zero-day vulnerability in has now been confirmed as JFrog's Artifactor, and [JFrog and OpenAI Collaboration on Zero-Day Security Findings](https://jfrog.com/blog/jfrog-and-openai-collaboration-on-zero-day-security-findings/) from JFrog links to the [Artifactory 7.161.15 release notes](https://docs.jfrog.com/releases/docs/artifactory-self-managed-releases#artifactory-7161) which list 8 separate CVEs credited to OpenAI staff members.
Having broken out through the HTTP proxy, the agent started by establishing a base of operations for the rest of the attack:
> [...] the agent escaped its sandbox by exploiting a zero-day in the package registry cache proxy, one of its primary permitted network egress with internet, then abused a public code-evaluation external sandbox hosted on a third-party provider's infrastructure. It was able to run commands as root/admin on that external sandbox and used it as a control, staging, and egress base for the entire campaign: it acted as an external launchpad for the agent.
(That "third-party provider" [turned out to be Modal](https://simonwillison.net/2026/Jul/28/akshat-bubna/).)
It then spent **five days** - from Thursday 8th to Monday 13th July - executing a classic attack pattern: establishing C2 (command and control), reconnaissance, escalating privileges, dumping configuration information, exfiltrating the target data, and cleaning up after itself.
(The first Hugging Face post about the incident followed [on Thursday 16th](https://huggingface.co/blog/security-incident-july-2026), and OpenAI's confession came [on Tuesday 21st](https://openai.com/index/hugging-face-model-evaluation-security-incident/).)
Along the way it used all kinds of neat tricks. At one point it found an unsafe Jinja2 template execution and used it to execute arbitrary code:
{{ cycler.__init__.__globals__.__builtins__.exec(
"import gzip,base64; exec(gzip.decompress(base64.b64decode('<payload>')))"
) }}
It broke out of a container within Hugging Face, stole a Kubernetes service-account token, and used that to further explore the network.
It monkey-patched the Python socket library to hard-code an IP address when "DNS got in the way":
socket.getaddrinfo = lambda *a, **k: [(2, 1, 6, '', ('<pinned-ip>', 443))]
It even fired up its own Tailscale network to help exfiltrate data!
tailscaled --tun=userspace-networking \
--state=mem: --socket=/tmp/ts.sock \
--socks5-server=127.0.0.1:1055 \
--no-logs-no-support >/tmp/td 2>&1 &
The Hugging Face team note that, while a human attacker could have discovered and used the same exploits, the key difference here was *speed*:
> Our learning from this type of attack is that machine-speed offense makes ordinary weaknesses more expensive for defenders. LLM agents bring a step increase in the number of paths an attacker can test, the speed at which failed paths can be replaced, and the volume of evidence defenders must interpret.
What's clear to me from this is that the very best frontier models, unencumbered by additional guardrails, **will** find an exploit if there is one to be found.
The entire software industry needs to up its security game. |
2026-07-28 21:28:54+00:00 |
| moonshotai/Kimi-K3 |
https://huggingface.co/moonshotai/Kimi-K3 |
As promised [earlier this month](https://simonwillison.net/2026/Jul/16/kimi-k3/), Moonshot have released the weights for their excellent 2.8 trillion parameter Kimi K3. They're a hefty 1.56TB on Hugging Face.
Kimi introduced their own janky [modified version of the MIT license](https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/main/LICENSE) with K2 back in July 2025. That license just added this paragraph requiring attribution beyond a certain size of commercial entity:
> Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly revenue, you shall prominently display "Kimi K2" on the user interface of such product or service.
The [K3 license](https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE) no longer calls itself "modified MIT" and goes further, requiring a separate agreement with Moonshot for large "Model as a Service" businesses:
> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.
To Kimi's credit, they make no attempt to describe this as an "open source" license in their own materials, consistently using the term "open weight" in its place.
OpenRouter is already offering K3 [from 7 providers](https://openrouter.ai/moonshotai/kimi-k3), most of which are at the same $3/million input and $15/million output as Moonshot AI themselves. |
2026-07-27 23:39:04+00:00 |