Simon Willison’s Weblog

Announcing Purple Llama: Towards open trust and safety in the new world of generative AI (via) New from Meta AI, Purple Llama is “an umbrella project featuring open trust and safety tools and evaluations meant to level the playing field for developers to responsibly deploy generative AI models and experiences”.

There are three components: a 27 page “Responsible Use Guide”, a new open model called Llama Guard and CyberSec Eval, “a set of cybersecurity safety evaluations benchmarks for LLMs”.

Disappointingly, despite this being an initiative around trustworthy LLM development,prompt injection is mentioned exactly once, in the Responsible Use Guide, with an incorrect description describing it as involving “attempts to circumvent content restrictions”!

The Llama Guard model is interesting: it’s a fine-tune of Llama 2 7B designed to help spot “toxic” content in input or output from a model, effectively an openly released alternative to OpenAI’s moderation API endpoint.

The CyberSec Eval benchmarks focus on two concepts: generation of insecure code, and preventing models from assisting attackers from generating new attacks. I don’t think either of those are anywhere near as important as prompt injection mitigation.

My hunch is that the reason prompt injection didn’t get much coverage in this is that, like the rest of us, Meta’s AI research teams have no idea how to fix it yet! # 8th December 2023, 6:36 am

Pysa: An open source static analysis tool to detect and prevent security issues in Python code (via) Interesting new static analysis tool for auditing Python for security vulnerabilities—things like SQL injection and os.execute() calls. Built by Facebook and tested extensively on Instagram, a multi-million line Django application. # 7th August 2020, 8:50 pm

What is a Self-XSS scam? Facebook link to this page from a console.log message that they display the browser devtools console, specifically warning that “If someone told you to copy-paste something here to enable a Facebook feature or hack someone’s account, it is a scam and will give them access to your Facebook account.” # 8th April 2019, 6:01 pm

Upgrades to Facebook’s link security (via) Facebook have started scanning links shared on the site for HSTS headers, which are used to indicate that an HTTP page is also available over HTTPS and are intended to be cached by browsers such that future HTTP access is automatically retrieved over HTTPS instead. Facebook will now obey those headers itself and link directly to the HTTPS version. What a great idea: all sites with sophisticated link sharing (where links are fetched to retrieve extracts and images for example) should do this as well. # 5th March 2018, 3:32 pm

The Inside Story of How Facebook Responded to Tunisian Hacks (via) “By January 5, it was clear that an entire country’s worth of passwords were in the process of being stolen right in the midst of the greatest political upheaval in two decades.”—which is why you shouldn’t serve your login form over HTTP even though it POSTs over HTTPS. # 24th January 2011, 6:06 pm

“Likejacking” Takes Off on Facebook. The Facebook Like button is vulnerable to Clickjacking, and is being widely exploited. Since Likes show up in your Facebook stream, it’s an easy attack to make viral. The button is implemented on third party sites as an iframe, which would seem to me to be exploitable by design (just make the iframe transparent in the parent document and trick the user in to clicking in the right place). I can’t think of any way they could support the embedded Like button without being vulnerable to clickjacking, since clickjacking prevention relies on not allowing your UI elements to be embedded in a hostile site while the Like button’s functionality depends on exactly that. # 3rd June 2010, 10:01 am

Facebook Adds Code for Clickjacking Prevention. Clever technique: Facebook pages check to see if they are being framed (using window.top) and, if they are, add a div covering the whole page which causes a top level reload should anything be clicked on. They also log framing attempts using an image bug. # 13th March 2010, 10:42 am

Some People Can’t Read URLs. Commentary on the recent “facebook login” incident from Jono at Mozilla Labs. I’d guess that most people can’t read URLs, and it worries me more than any other aspect of today’s web. If you want to stay safe from phishing and other forms of online fraud you need at least a basic understanding of a bewildering array of technologies—URLs, paths, domains, subdomains, ports, DNS, SSL as well as fundamental concepts like browsers, web sites and web servers. Misunderstand any of those concepts and you’ll be an easy target for even the most basic phishing attempts. It almost makes me uncomfortable encouraging regular people to use the web because I know they’ll be at massive risk to online fraud. # 2nd March 2010, 10:16 am

The Dangers of Clickjacking with Facebook. theharmonyguy compiled a list of actions that can be triggered on Facebook by a single click, and hence are vulnerable to clickjacking attacks. The list includes authorising malicious applications, posting links to profiles, sending friend requests and sending messages to other users. Why don’t Facebook include frame busting JavaScript on every page? # 23rd December 2009, 10:20 am

New Facebook clickjacking attack in the wild. I’m not sure why Facebook don’t use frame-busting JavaScript to avoid this kind of thing. The attack is pretty crafty—a Facebook page is positioned with everything obscured bar part of the blue “share this” button, and a fake “Human Test” asks the user to find and click the blue button to continue. # 22nd December 2009, 6:52 pm

Facebook and MySpace security: backdoor wide open, millions of accounts exploitable (via) Amazingly, both services had wide open holes in their crossdomain.xml files. Facebook were serving allow-access-from-domain=“*” in the crossdomain.xml file on one of their subdomains (a subdomain that still had access to the user’s profile information) while MySpace were opting in farm.sproutbuilder.com, a service which allowed anyone to upload arbitrary SWF files. # 5th November 2009, 9:47 am

Facebook Hacked By 4chan, Accounts Compromised. It wasn’t Facebook that got hacked: 4chan members got hold of a list of usernames and passwords from an insecure Christian dating site and started using them to raise complete hell. Yet another demonstration that storing your user’s passwords in the clear is extremely irresponsible, and also a handy reminder that regular users who “don’t have anything worth securing” actually have a great deal to lose if their password gets out. # 23rd August 2009, 10:02 am

FB App Canvas Pages: I Think I’d Use IFrames. Facebook’s Charlie Cheever explains the difference between FBML canvas pages, iframe pages and XFBML when building Facebook apps. I’m always surprised at APIs that load untrusted content in an iframe, as it seems like an invitation for frame-busting phishing attacks. # 2nd October 2008, 2:39 pm

Facebook Security Advice: Never Ever Enter Your Passwords On Another Site, Unless We Ask You To. Nice to see TechCrunch highlighting the hypocrisy of Facebook advising their users to never enter their Facebook credentials on another site, then asking them for their webmail provider password so they can scrape their address book. # 9th August 2008, 10:18 am

Top XSS exploits by PageRank. Yahoo!, MSN, Google, YouTube, MySpace, FaceBook all feature. # 29th May 2007, 10:07 pm