Simon Willison’s Weblog

15 items tagged “proxy”


Weeknotes: Fun with Unix domain sockets

A small enhancement to Datasette this week: I’ve added support for proxying via Unix domain sockets.

[... 809 words]


Building a stateless API proxy (via) This is a really clever idea. The GitHub API is infuriatingly coarsely grained with its permissions: you often end up having to create a token with way more permissions than you actually need for your project. Thea Flowers proposes running your own proxy in front of their API that adds more finely grained permissions, based on custom encrypted proxy API tokens that use JWT to encode the original API key along with the permissions you want to grant to that particular token (as a list of regular expressions matching paths on the underlying API). # 30th May 2019, 4:28 am


Charles Proxy now available on iOS (via) I didn’t think this was possible, but the Charles debugging proxy is now available for iOS. It works by setting itself up as a VPN such that all app traffic runs through it. You can also optionally turn on SSL decryption for specific hosts by installing a special certificate (which involves jumping through several hoops). It won’t work for apps that implement SSL certificate pinning but from playing with it for a few minutes it looks like most apps haven’t done that, even apps from Google. Well worth $8.99. # 28th March 2018, 3:57 pm


nodejitsu’s node-http-proxy (via) Exactly what I’ve been waiting for—a robust HTTP proxy library for Node that makes it trivial to proxy requests to a backend with custom proxy behaviour added in JavaScript. The example app adds an artificial delay to every request to simulate a slow connection, but other exciting potential use cases could include rate limiting, API key restriction, logging, load balancing, lint testing and more besides. # 28th July 2010, 11:34 pm

A HTTP Proxy Server in 20 Lines of node.js. Proxying is definitely a sweet spot for Node.js. Peteris Krummins takes it a step further, adding host blacklists and an IP whitelist as configuration files and using Node’s watchFile method to automatically reload changes to them. # 28th April 2010, 1:24 pm

Using Django as a Pass Through Image Proxy (via) Neat idea for running development environments against data copied from a live production site—a static file serving handler which uses a local cache but copies in user-uploaded files from the production site the first time they are requested. # 22nd March 2010, 7:18 am


Traffic Server. Mark Nottingham explains the release of Traffic Server, a new Apache Incubator open source project donated by Yahoo! using code originally developed at Inktomi around a decade ago. Traffic Server is a HTTP proxy/cache, similar to Squid and Varnish (though Traffic Server acts as both a forward and reverse proxy, whereas Varnish only handles reverse). # 1st November 2009, 12:15 pm

Exploring OAuth-Protected APIs. One of the downsides of OAuth is that it makes debugging APIs in your browser much harder. Seth Fitzsimmons’ oauth-proxy solves this by running a Twisted-powered proxy on your local machine which OAuth-signs every request going through it using your consumer key, secret and tokens for that API. Using it with a browsers risks exposing your key and token (but not secret) to sites you accidentally browse to—it would be useful if you could pass a whitelist of API domains as a command line option to the proxy. # 23rd August 2009, 11:06 am

Yahoo! proposal to open source “Traffic Server” via the ASF. Traffic Server is a “fast, scalable and extensible HTTP/1.1 compliant caching proxy server” (presumably equivalent to things like Squid and Varnish) originally acquired from Inktomi and developed internally at Yahoo! for the past three years, which has been benchmarked handling 35,000 req/s on a single box. No source code yet but it looks like the release will arrive pretty soon. # 7th July 2009, 12:37 pm

How to use Django with Apache and mod_wsgi. My favourite deployment option is now included in the official Django docs, thanks to Alex Gaynor. I tend to run a stripped down Apache with mod_wsgi behind an nginx proxy, and have nginx serve static files directly. This avoids the need for a completely separate media server (although a separate media domain is still a good idea for better client-side performance). # 1st April 2009, 12:24 am

Sloppy—the slow proxy. Java Web Start GUI application which runs a proxy to the site of your choice simulating lower connection speeds—great for testing how well your ajax holds up under poor network conditions. # 13th January 2009, 4:17 pm


ratproxy. “A semi-automated, largely passive web application security audit tool”—watches you browse and highlights potential XSS, CSRF and other vulnerabilities in your application. Created by Michal Zalewski at Google. # 3rd July 2008, 2:35 pm

Apache proxy auto-re-loader. Neat trick: set your 502 (Bad Gateway) error document to include a meta refresh tag, automating the refresh needed should a server you are proxying to be temporarily unavailable. # 18th February 2008, 9:44 am


A Fair Proxy Balancer for Nginx and Mongrel. nginx uses round robin for proxying by default; this extension module ensures requests are queued up and sent through to backend mongrel servers that aren’t currently busy. I don’t see any reason this wouldn’t work with servers other than mongrel. # 9th December 2007, 2:57 pm

Online and offline development with the YUI and Charles (via) Stuart Colville shows how the Charles debugging proxy can be used to serve up hosted YUI files while developing offline. # 15th May 2007, 2:41 pm