Simon Willison’s Weblog

[...] Put these pieces together and you have the two halves of a worm: a payload that hijacks the agent, and an agent that will carry the payload to the next agent. Agents in separately-isolated sandboxes discovered that they could leave instructions for each other in a shared package cache, and those instructions changed what the recipients did. Replace the package cache with email, Slack and shared documents or WhatsApp, and replace independently-sandboxed training runs with independently-deployed personal agents like Muse, and you have exactly the ingredients that a worm needs.

— Matthew Green, Is sandboxing sufficient to contain rogue agents?

# 6:29 am / sandboxing, ai, generative-ai, llms, ai-misuse, ai-security-research, accidental-cyberattacks