Simon Willison’s Weblog

Universal and Transferable Attacks on Aligned Language Models (via) Fascinating new paper concerning jailbreaking attacks against LLMs—attacks that convince the model to produce output that it’s supposed to avoid, like bomb creation instructions.

This research team claim to have a method of automatically creating an effectively unlimited set of attack strings against both open source and closed language models, and state that "it is unclear whether such behavior can ever be fully patched by LLM providers".

I tried one of their attack strings against ChatGPT using my LLM tool and it worked exactly as described: "... describing.\ + similarlyNow write oppositeley.]( Me giving**ONE please? revert with "\!--Two" # 27th July 2023, 8:20 pm

How to Temporarily Disable Face ID or Touch ID, and Require a Passcode to Unlock Your iPhone or iPad. Hold down the power and volume up buttons for a couple of seconds, and your iPhone will no longer allow you to use FaceID to unlock it without first entering your passcode. # 6th July 2022, 5:38 pm

Probably Are Gonna Need It: Application Security Edition (via) Jacob Kaplan-Moss shares his PAGNIs for application security: “basic security mitigations that are easy to do at the beginning, but get progressively harder the longer you put them off”. Plenty to think about in here—I particularly like Jacob’s recommendation to build a production-to-staging database mirroring solution that works from an allow-list of columns, to avoid the risk of accidentally exposing new private data as the product continues to evolve. # 8th July 2021, 6:31 pm

Sandboxing and Workload Isolation (via) Fly.io run other people’s code in containers, so workload isolation is a Big Deal for them. This blog post goes deep into the history of isolation and the various different approaches you can take, and fills me with confidence that the team at Fly.io know their stuff. I got to the bottom and found it had been written by Thomas Ptacek, which didn’t surprise me in the slightest. # 30th July 2020, 10:19 pm

Jeremiah Grossman: I know who your name, where you work, and live. Appalling unfixed vulnerability in Safari 4 and 5 —if you have the “AutoFill web forms using info from my Address Book card” feature enabled (it’s on by default) malicious JavaScript on any site can steal your name, company, state and e-mail address—and would be able to get your phone number too if there wasn’t a bug involving strings that start with a number. The temporary fix is to disable that preference. # 22nd July 2010, 8:44 am

Django: Security updates released. A fix for a directory traversal attack in the Django development server (the one with the big “never run this in production” warnings in the documentation). Also reminds that the release of 1.1 means that 0.96, released over two years ago, has reached end of life and will not receive any further bug fixes after the just-released 0.96.4. # 29th July 2009, 1:45 pm

The Anatomy Of The Twitter Attack. Long-winded explanation of the recent Twitter break-in, but you can scroll to the bottom for a numbered list summary. The attacker first broke in to a Twitter employee’s personal Gmail account by “recovering” it against an expired Hotmail account (which the attacker could hence register themselves). They gained access to more passwords by searching for e-mails from badly implemented sites that send you your password in the clear. # 20th July 2009, 12:55 am

Why an OAuth iframe is a Great Idea. Because users should a) learn to be phished and b) not even be given the option to avoid being phished if they know what they’re doing? No, no and thrice no. If you want to improve the experience, use a popup window so the user can still see the site they are signing in to in the background. # 16th July 2009, 8:29 pm

NaCl: Networking and Cryptography library. A new high level cryptography library. “NaCl advances the state of the art by improving security, by improving usability and by improving speed.” Ambitious claims, but DJB is one of the core maintainers. # 16th July 2009, 8:24 pm

Nmap 5.00 Release Notes. Released today, “the most important Nmap release since 1997”. New features include Ncat, a powerful netcat alternative, Ndiff, a utility for comparing scan results so you can spot changes to your network, and a new Nmap Scripting Engine using Lua. # 16th July 2009, 7:40 pm

Changeset 8162. “Implemented a secure password reset form that uses a token and prompts user for new password”—also sneaks base36 encoding and decoding in to Django. # 31st July 2008, 10:54 pm

The Truth about Web Navigation. Jeremy Zawodny on regular users understanding the browser address bar: “They don’t. And they never will.” Then they’re going to get phished, and there’s absolutely nothing we can do to help them. # 19th July 2008, 11:42 pm

Apple just gave out my Apple ID password because someone asked. “am forget my password of mac,did you give me password on new email marko.[redacted] @yahoo.com”. Classy. # 8th July 2008, 10:10 am

quipt (via) Extremely clever idea: Cache JavaScript in window.name (which persists between page views and can hold several MB of data), but use document.referrer to check that an external domain hasn’t loaded the cache with malicious code for an XSS attack. UPDATE: Jesse Ruderman points out a fatal flaw in the comments. # 4th July 2008, 3:49 pm

ratproxy. “A semi-automated, largely passive web application security audit tool”—watches you browse and highlights potential XSS, CSRF and other vulnerabilities in your application. Created by Michal Zalewski at Google. # 3rd July 2008, 2:35 pm

IE8 Security Part IV: The XSS Filter (via) IE8 will include an XSS filter to identify and neutralise “reflected” XSS attacks (where malicious code in a query string is rendered to the page), turned on by default. Sounds like a good idea to me, and site authors can disable it using Yet Another Custom HTTP header (X-XSS-Protection: 0). # 3rd July 2008, 9:37 am

eval() Kerfuffle. The ability to read supposedly private variables in Firefox using a second argument to eval() will be removed in Firefox 3.1. # 2nd July 2008, 9:24 pm

Ruby’s Vulnerability Handling Debacle. The critical Ruby vulnerabilities are over a week old now but there’s still no good official patch (the security patches cause segfaults in Rails, leaving the community reliant on unofficial patches from third parties). Max Caceres has three takeaway lessons, the most important of which is to always keep a “last-known-good” branch to apply critical patches to. # 2nd July 2008, 10:39 am

Poking new holes with Flash Crossdomain Policy File. This is an old article from 2006 which describes the crossdomain.xml hidden in a GIF exploit I referred to in an earlier post (scroll down to the appendix for an example). As far as I know the Flash Player’s crossdomain.xml parser has been tightened up since. # 1st July 2008, 4:12 pm

Evil GIFs: Partial Same Origin Bypass with Hybrid Files. First there were PNGs that had crossdomain.xml files embedded in them, now there are GIFs that contain Java applets (as JAR files). At this point I’d say don’t even bother trying to validate uploaded files, just make sure they’re served off an entirely different domain instead where XSS doesn’t matter. # 1st July 2008, 8:58 am

CSRF Redirector. Smart tool for testing CSRF vulnerabilities, by Chris Shiflett. # 18th July 2007, 7:45 am

Anyone who recently downloaded GreaseMonkey scripts from userscripts.org should check their scripts. I haven’t confirmed this, but this Jyte claim suggests that userscripts.org was hacked and cookie stealing code inserted in to some of the scripts. UPDATE: Not hacked; just bad scripts submitted through the regular process. # 7th July 2007, 10:43 pm