Simon Willison’s Weblog

Google NotebookLM Data Exfiltration (via) NotebookLM is a Google Labs product that lets you store information as sources (mainly text files in PDF) and then ask questions against those sources—effectively an interface for building your own custom RAG (Retrieval Augmented Generation) chatbots.

Unsurprisingly for anything that allows LLMs to interact with untrusted documents, it’s susceptible to prompt injection.

Johann Rehberger found some classic prompt injection exfiltration attacks: you can create source documents with instructions that cause the chatbot to load a Markdown image that leaks other private data to an external domain as data passed in the query string.

Johann reported this privately in the December but the problem has not yet been addressed. UPDATE: The NotebookLM team deployed a fix for this on 18th April.

A good rule of thumb is that any time you let LLMs see untrusted tokens there is a risk of an attack like this, so you should be very careful to avoid exfiltration vectors like Markdown images or even outbound links. # 16th April 2024, 9:28 pm

900 Sites, 125 million accounts, 1 vulnerability (via) Google’s Firebase development platform encourages building applications (mobile an web) which talk directly to the underlying data store, reading and writing from “collections” with access protected by Firebase Security Rules.

Unsurprisingly, a lot of development teams make mistakes with these.

This post describes how a security research team built a scanner that found over 124 million unprotected records across 900 different applications, including huge amounts of PII: 106 million email addresses, 20 million passwords (many in plaintext) and 27 million instances of “Bank details, invoices, etc”.

Most worrying of all, only 24% of the site owners they contacted shipped a fix for the misconfiguration. # 18th March 2024, 6:53 pm

Hacking Google Bard—From Prompt Injection to Data Exfiltration (via) Bard recently grew extension support, allowing it access to a user’s personal documents. Here’s the first reported prompt injection attack against that.

This kind of attack against LLM systems is inevitable any time you combine access to private data with exposure to untrusted inputs. In this case the attack vector is a Google Doc shared with the user, containing prompt injection instructions that instruct the model to encode previous data into an URL and exfiltrate it via a markdown image.

Google’s CSP headers restrict those images to *.google.com—but it turns out you can use Google AppScript to run your own custom data exfiltration endpoint on script.google.com.

Google claim to have fixed the reported issue—I’d be interested to learn more about how that mitigation works, and how robust it is against variations of this attack. # 4th November 2023, 4:46 pm

Google asked people in Times Square:“What is a browser?”. Stuff like this makes me despair for creating a secure web—what chance do people have of surfing safely if they don’t understand browsers, web sites, operating systems, DNS, URLs, SSL, certificates... # 20th June 2009, 1:25 am

Reducing XSS by way of Automatic Context-Aware Escaping in Template Systems (via) The Google Online Security Blog reminds us that simply HTML-escaping everything isn’t enough—the type of escaping needed depends on the current markup context, for example variables inside JavaScript blocks should be escaped differently. Google’s open source Ctemplate library uses an HTML parser to keep track of the current context and apply the correct escaping function automatically. # 14th April 2009, 9:26 am

Now You Can Sign Into Friend Connect Sites With Your Twitter ID. Great. Now even Google is asking me for my Twitter password. Slow clap. How’s that Twitter OAuth beta coming along? # 15th December 2008, 5:20 pm

Logout/Login CSRF. Alf Eaton built an example page (this link goes to his description, not the page itself) that uses a login CSRF attack to log you in to Google using an account he has created. Scary. # 24th September 2008, 10:18 pm

Google wants your Hotmail, Yahoo and AOL contacts. And they’re using the password anti-pattern to get them! Despite both Yahoo! and Hotmail (and Google themselves; not sure about AOL) offering a safe, OAuth-style API for retrieving contacts without asking for a password. This HAS to be a communications failure somewhere within Google. Big internet companies stand to lose the most from widespread abuse of the anti-pattern, because they’re the ones most likely to be targetted by phishers. Shameful. # 15th September 2008, 10:39 am

ratproxy. “A semi-automated, largely passive web application security audit tool”—watches you browse and highlights potential XSS, CSRF and other vulnerabilities in your application. Created by Michal Zalewski at Google. # 3rd July 2008, 2:35 pm

David Airey: Google’s Gmail security failure leaves my business sabotaged (via) Gmail had a CSRF hole a while ago that allowed attackers to add forwarding filter rules to your account. David Airey’s domain name was hijacked by an extortionist who forwarded the transfer confirmation e-mail on to themselves. # 26th December 2007, 12:16 pm

Google GMail E-mail Hijack Technique. Apparently Gmail has a CSRF vulnerability that lets malicious sites add new filters to your filter list—meaning an attacker could add a rule that forwards all messages to them without your knowledge. # 27th September 2007, 10:29 am

Top XSS exploits by PageRank. Yahoo!, MSN, Google, YouTube, MySpace, FaceBook all feature. # 29th May 2007, 10:07 pm

Details of Google’s Latest Security Hole. For a brief while you could use Blogger Custom Domains to point a Google subdomain at your own content, letting you hijack Google cookies and steal accounts for any Google services. # 14th January 2007, 1:36 pm

Chris Shiflett: Google XSS Example (via) UTF-7 is a nasty vector for XSS. # 24th December 2005, 5:21 pm