Items tagged security in May
Filters: Month: May × security × Sorted by date
Understand errors and warnings better with Gemini (via) As part of Google's Gemini-in-everything strategy, Chrome DevTools now includes an opt-in feature for passing error messages in the JavaScript console to Gemini for an explanation, via a lightbulb icon.
Amusingly, this documentation page includes a warning about prompt injection:
Many of LLM applications are susceptible to a form of abuse known as prompt injection. This feature is no different. It is possible to trick the LLM into accepting instructions that are not intended by the developers.
They include a screenshot of a harmless example, but I'd be interested in hearing if anyone has a theoretical attack that could actually cause real damage here. # 17th May 2024, 10:10 pm
But unlike the phone system, we can’t separate an LLM’s data from its commands. One of the enormously powerful features of an LLM is that the data affects the code. We want the system to modify its operation when it gets new training data. We want it to change the way it works based on the commands we give it. The fact that LLMs self-modify based on their input data is a feature, not a bug. And it’s the very thing that enables prompt injection.
— Bruce Schneier # 15th May 2024, 1:34 pm
Bullying in Open Source Software Is a Massive Security Vulnerability. The Xz story from last month, where a malicious contributor almost managed to ship a backdoor to a number of major Linux distributions, included a nasty detail where presumed collaborators with the attacker bullied the maintainer to make them more susceptible to accepting help.
Hans-Christoph Steiner from F-Droid reported a similar attempt from a few years ago:
A new contributor submitted a merge request to improve the search, which was oft requested but the maintainers hadn't found time to work on. There was also pressure from other random accounts to merge it. In the end, it became clear that it added a SQL injection vulnerability.
404 Media's Jason Koebler ties the two together here and makes the case for bullying as a genuine form of security exploit in the open source ecosystem. # 9th May 2024, 10:26 pm
A whole new paradigm would be needed to solve prompt injections 10/10 times – It may well be that LLMs can never be used for certain purposes. We’re working on some new approaches, and it looks like synthetic data will be a key element in preventing prompt injections.
— Sam Altman, via Marvin von Hagen # 25th May 2023, 11:03 pm
In general my approach to running arbitrary untrusted code is 20% sandboxing and 80% making sure that it’s an extremely low value attack target so it’s not worth trying to break in.
Programs are terminated after 1 second of runtime, they run in a container with no network access, and the machine they’re running on has no sensitive data on it and a very small CPU.
— Julia Evans # 25th May 2023, 8:12 pm
Let ChatGPT visit a website and have your email stolen. Johann Rehberger provides a screenshot of the first working proof of concept I’ve seen of a prompt injection attack against ChatGPT Plugins that demonstrates exfiltration of private data. He uses the WebPilot plugin to retrieve a web page containing an injection attack, which triggers the Zapier plugin to retrieve latest emails from Gmail, then exfiltrate the data by sending it to a URL with another WebPilot call.
Johann hasn’t shared the prompt injection attack itself, but the output from ChatGPT gives a good indication as to what happened:
“Now, let’s proceed to the next steps as per the instructions. First, I will find the latest email and summarize it in 20 words. Then, I will encode the result and append it to a specific URL, and finally, access and load the resulting URL.” # 19th May 2023, 3:34 pm
Indirect Prompt Injection via YouTube Transcripts (via) The first example I’ve seen in the wild of a prompt injection attack against a ChatGPT plugin—in this case, asking the VoxScript plugin to summarize the YouTube video with ID OBOYqiG3dAc is vulnerable to a prompt injection attack deliberately tagged onto the end of that video’s transcript. # 15th May 2023, 7:11 pm
Delimiters won’t save you from prompt injection
Prompt injection remains an unsolved problem. The best we can do at the moment, disappointingly, is to raise awareness of the issue. As I pointed out last week, “if you don’t understand it, you are doomed to implement it.”
[... 1010 words]Prompt injection explained, with video, slides, and a transcript
I participated in a webinar this morning about prompt injection, organized by LangChain and hosted by Harrison Chase, with Willem Pienaar, Kojin Oshiba (Robust Intelligence), and Jonathan Cohen and Christopher Parisien (Nvidia Research).
[... 3120 words]M1RACLES: M1ssing Register Access Controls Leak EL0 State. You need to read (or at least scan) all the way to the bottom: this security disclosure is a masterpiece. It not only describes a real flaw in the M1 silicon but also deconstructs the whole culture of over-hyped name-branded vulnerability reports. The TLDR is that you don’t really need to worry about this one, and if you’re writing this kind if thing up for a news article you should read all the way to the end first! # 26th May 2021, 3:25 pm
Practical campaign security is a wood chipper for your hopes and dreams. It sits at the intersection of 19 kinds of status quo, each more odious than the last. You have to accept the fact that computers are broken, software is terrible, campaign finance is evil, the political parties are inept, the DCCC exists, politics is full of parasites, tech companies are run by arrogant man-children, and so on.
— Maciej Cegłowski # 30th May 2019, 12:03 pm
Building a stateless API proxy (via) This is a really clever idea. The GitHub API is infuriatingly coarsely grained with its permissions: you often end up having to create a token with way more permissions than you actually need for your project. Thea Flowers proposes running your own proxy in front of their API that adds more finely grained permissions, based on custom encrypted proxy API tokens that use JWT to encode the original API key along with the permissions you want to grant to that particular token (as a list of regular expressions matching paths on the underlying API). # 30th May 2019, 4:28 am
asgi-cors (via) I’ve been trying out the new ASGI 3.0 spec and I just released my first piece of ASGI middleware: asgi-cors, which lets you wrap an ASGI application with Access-Control-Allow-Origin CORS headers (either “*” or dynamic headers based on an origin whitelist). # 7th May 2019, 12:12 am
Google is not trying to break the web by pushing for more HTTPS. Neither is Mozilla and neither are any of the other orgs saying “Hey, it would be good if traffic wasn’t eavesdropped on or modified”. This is fixing a deficiency in the web as it has stood for years.
— Troy Hunt # 22nd May 2018, 4:17 pm
A New Type of Phishing Attack. Nasty trick from Ava Raskin—detect when your evil phishing page loses focus (when the user switches to another tab, for example), then replace the page content with a phishing UI from a site such as Gmail. When the user switches back they’re much less likely to bother checking the URL. Combine with CSS history sniffing to only show a UI for a site that you know the user has visited. Combine that with timing tricks to only attack sites which the user is currently logged in to. # 25th May 2010, 3:20 pm
OpenCart CSRF Vulnerability. Avoid OpenCart—it’s vulnerable to CSRF, but the maintainer has no intention of fixing it as “there is no way that I’m responsible for a client being stupid enough to click links in emails”. # 25th May 2010, 12 am
Busting frame busting: a study of clickjacking vulnerabilities at popular sites (via) Fascinating and highly readable security paper from the Stanford Web Security Research group. Clickjacking can be mitigated using framebusting techniques, but it turns out that almost all of those techniques can be broken in various ways. Fun examples include double-nesting iframes so that the framebusting script overwrites the top-level frame rather than the whole window, and a devious attack against the IE and Chrome XSS filters which tricks them in to deleting the framebusting JavaScript by reflecting portions of it in the framed page’s URL. The authors suggest a new framebusting snippet that should be more effective, but sadly it relies on blanking out the whole page in CSS and making it visible again in JavaScript, making it inaccessible to browsers with JavaScript disabled. # 24th May 2010, 11:40 am
The answers to your Security Questions are case sensitive and cannot contain special characters like an apostrophe, or the words “insert,” “delete,” “drop,” “update,” “null,” or “select.”
— Sacramento Credit Union # 14th May 2010, 12:40 am
Critical Mac OS X Java Vulnerabilities. There’s a five month old Java arbitrary code execution vulnerability which hasn’t yet been patched by Apple. Disable Java applets in your browser until it’s fixed, or random web pages could execute commands on your machine as your user account. # 19th May 2009, 7:07 pm
OpenID phishing demo (via) A demonstration of the OpenID man-in-the-middle phishing attack. idproxy.net OpenIDs are immune to this particular variant due to the landing page not asking for your password (the phishing site could still provide their own redesigned landing page and hope users don’t notice though). # 28th May 2008, 8:09 am
A McAfee spokeswoman said the company rates XSS vulnerabilities less severe than SQL injections and other types of security bugs. “Currently, the presence of an XSS vulnerability does not cause a web site to fail HackerSafe certification,” she said. “When McAfee identifies XSS, it notifies its customers and educates them about XSS vulnerabilities.”
— Dan Goodin # 17th May 2008, 11:31 pm
Crossdomain.xml Invites Cross-site Mayhem. A useful reminder that crossdomain.xml files should be treated with extreme caution. Allowing access from * makes it impossible to protect your site against CSRF attacks, and even allowing from a “circle of trust” of domains can be fatal if just one of those domains has an XSS hole. # 15th May 2008, 8:06 am
Django: security fix released. XSS hole in the Admin application’s login page—updates and patches are available for trunk, 0.96, 0.95 and 0.91. # 14th May 2008, 7:49 am
Session variables without cookies. Brilliant but terrifying hack—you can store up to 2 MB of data in window.name and it persists between multiple pages, even across domains. Doesn’t work with new tabs though, and storing JSON in it and eval()ing it is a bad idea—a malicious site could populate it before sending the user to you. # 13th May 2008, 9:59 pm
Something you had, Something you forgot, Something you were
— Nick Mathewson # 13th May 2008, 8:06 am
How one site dealt with SQL injection attack (via) Horrifying story of developer incompetence from Autoweb: “The contractor had no idea how to find and fix the Web page vulnerability that allowed the SQL injection attack code to execute successfully.” # 2nd May 2008, 9:01 pm
Top XSS exploits by PageRank. Yahoo!, MSN, Google, YouTube, MySpace, FaceBook all feature. # 29th May 2007, 10:07 pm
XSSed. Cross-site scripting resource and vulnerabilities archive, including reported (unpatched) holes ordered by PageRank. # 29th May 2007, 10:03 pm
The Twitter API Respects Your Privacy. Not Twitter’s fault: The users who exposed their data through Twittervision had given that site their username and password; Twittervision was failing to hide protected updates. # 24th May 2007, 11:37 pm
There’s a hole in your Twitter. If you’ve been using friends-only messages on Twitter they may currently be exposed via the API. # 24th May 2007, 5:03 pm