Simon Willison’s Weblog

Subscribe
Atom feed for proxies

19 posts tagged “proxies”

2025

httpjail (via) Here's a promising new (experimental) project in the sandboxing space from Ammar Bandukwala at Coder. httpjail provides a Rust CLI tool for running an individual process against a custom configured HTTP proxy.

The initial goal is to help run coding agents like Claude Code and Codex CLI with extra rules governing how they interact with outside services. From Ammar's blog post that introduces the new tool, Fine-grained HTTP filtering for Claude Code:

httpjail implements an HTTP(S) interceptor alongside process-level network isolation. Under default configuration, all DNS (udp:53) is permitted and all other non-HTTP(S) traffic is blocked.

httpjail rules are either JavaScript expressions or custom programs. This approach makes them far more flexible than traditional rule-oriented firewalls and avoids the learning curve of a DSL.

Block all HTTP requests other than the LLM API traffic itself:

$ httpjail --js "r.host === 'api.anthropic.com'" -- claude "build something great"

I tried it out using OpenAI's Codex CLI instead and found this recipe worked:

brew upgrade rust
cargo install httpjail # Drops it in `~/.cargo/bin`
httpjail --js "r.host === 'chatgpt.com'" -- codex

Within that Codex instance the model ran fine but any attempts to access other URLs (e.g. telling it "Use curl to fetch simonwillison.net)" failed at the proxy layer.

This is still at a really early stage but there's a lot I like about this project. Being able to use JavaScript to filter requests via the --js option is neat (it's using V8 under the hood), and there's also a --sh shellscript option which instead runs a shell program passing environment variables that can be used to determine if the request should be allowed.

At a basic level it works by running a proxy server and setting HTTP_PROXY and HTTPS_PROXY environment variables so well-behaving software knows how to route requests.

It can also add a bunch of other layers. On Linux it sets up nftables rules to explicitly deny additional network access. There's also a --docker-run option which can launch a Docker container with the specified image but first locks that container down to only have network access to the httpjail proxy server.

It can intercept, filter and log HTTPS requests too by generating its own certificate and making that available to the underlying process.

I'm always interested in new approaches to sandboxing, and fine-grained network access is a particularly tricky problem to solve. This looks like a very promising step in that direction - I'm looking forward to seeing how this project continues to evolve.

# 19th September 2025, 9:57 pm / http, javascript, proxies, sandboxing, security, v8, rust, claude-code, codex-cli

2021

Weeknotes: Apache proxies in Docker containers, refactoring Datasette

Updates to six major projects this week, plus finally some concrete progress towards Datasette 1.0.

[... 1,630 words]

Weeknotes: Fun with Unix domain sockets

A small enhancement to Datasette this week: I’ve added support for proxying via Unix domain sockets.

[... 809 words]

2019

Building a stateless API proxy (via) This is a really clever idea. The GitHub API is infuriatingly coarsely grained with its permissions: you often end up having to create a token with way more permissions than you actually need for your project. Thea Flowers proposes running your own proxy in front of their API that adds more finely grained permissions, based on custom encrypted proxy API tokens that use JWT to encode the original API key along with the permissions you want to grant to that particular token (as a list of regular expressions matching paths on the underlying API).

# 30th May 2019, 4:28 am / apis, encryption, github, proxies, security, jwt

2018

Charles Proxy now available on iOS (via) I didn’t think this was possible, but the Charles debugging proxy is now available for iOS. It works by setting itself up as a VPN such that all app traffic runs through it. You can also optionally turn on SSL decryption for specific hosts by installing a special certificate (which involves jumping through several hoops). It won’t work for apps that implement SSL certificate pinning but from playing with it for a few minutes it looks like most apps haven’t done that, even apps from Google. Well worth $8.99.

# 28th March 2018, 3:57 pm / charles, proxies, ios

2010

Velocity: Forcing Gzip Compression. Almost every browser supports gzip these days, but 15% of web requests have had their Accept-Encoding header stripped or mangled, generally due to poorly implemented proxies or anti-virus software. Steve Souders passes on a trick used by Google Search, where an iframe is used to test the browser’s gzip support and set a cookie to force gzipping of future pages.

# 30th September 2010, 5:45 pm / browsers, gzip, performance, proxies, steve-souders, recovered

nodejitsu’s node-http-proxy (via) Exactly what I’ve been waiting for—a robust HTTP proxy library for Node that makes it trivial to proxy requests to a backend with custom proxy behaviour added in JavaScript. The example app adds an artificial delay to every request to simulate a slow connection, but other exciting potential use cases could include rate limiting, API key restriction, logging, load balancing, lint testing and more besides.

# 28th July 2010, 11:34 pm / http, javascript, node, nodejs, proxies, recovered

A HTTP Proxy Server in 20 Lines of node.js. Proxying is definitely a sweet spot for Node.js. Peteris Krummins takes it a step further, adding host blacklists and an IP whitelist as configuration files and using Node’s watchFile method to automatically reload changes to them.

# 28th April 2010, 1:24 pm / http, javascript, node, nodejs, peteris-krummins, proxies

Using Django as a Pass Through Image Proxy (via) Neat idea for running development environments against data copied from a live production site—a static file serving handler which uses a local cache but copies in user-uploaded files from the production site the first time they are requested.

# 22nd March 2010, 7:18 am / django, proxies

2009

Traffic Server. Mark Nottingham explains the release of Traffic Server, a new Apache Incubator open source project donated by Yahoo! using code originally developed at Inktomi around a decade ago. Traffic Server is a HTTP proxy/cache, similar to Squid and Varnish (though Traffic Server acts as both a forward and reverse proxy, whereas Varnish only handles reverse).

# 1st November 2009, 12:15 pm / apache, cache, http, inktomi, mark-nottingham, open-source, proxies, squid, trafficserver, varnish, yahoo

Exploring OAuth-Protected APIs. One of the downsides of OAuth is that it makes debugging APIs in your browser much harder. Seth Fitzsimmons’ oauth-proxy solves this by running a Twisted-powered proxy on your local machine which OAuth-signs every request going through it using your consumer key, secret and tokens for that API. Using it with a browsers risks exposing your key and token (but not secret) to sites you accidentally browse to—it would be useful if you could pass a whitelist of API domains as a command line option to the proxy.

# 23rd August 2009, 11:06 am / apis, oauth, proxies, python, seth-fitzsimmons, twisted

Yahoo! proposal to open source “Traffic Server” via the ASF. Traffic Server is a “fast, scalable and extensible HTTP/1.1 compliant caching proxy server” (presumably equivalent to things like Squid and Varnish) originally acquired from Inktomi and developed internally at Yahoo! for the past three years, which has been benchmarked handling 35,000 req/s on a single box. No source code yet but it looks like the release will arrive pretty soon.

# 7th July 2009, 12:37 pm / apache, asf, caching, open-source, proxies, squid, trafficserver, varnish, yahoo

How to use Django with Apache and mod_wsgi. My favourite deployment option is now included in the official Django docs, thanks to Alex Gaynor. I tend to run a stripped down Apache with mod_wsgi behind an nginx proxy, and have nginx serve static files directly. This avoids the need for a completely separate media server (although a separate media domain is still a good idea for better client-side performance).

# 1st April 2009, 12:24 am / alex-gaynor, deployment, django, modwsgi, nginx, proxies, python, wsgi

Sloppy—the slow proxy. Java Web Start GUI application which runs a proxy to the site of your choice simulating lower connection speeds—great for testing how well your ajax holds up under poor network conditions.

# 13th January 2009, 4:17 pm / ajax, java, javascript, javawebstart, performance, proxies, richard-dallaway, sloppy

2008

ratproxy. “A semi-automated, largely passive web application security audit tool”—watches you browse and highlights potential XSS, CSRF and other vulnerabilities in your application. Created by Michal Zalewski at Google.

# 3rd July 2008, 2:35 pm / csrf, google, michal-zalewski, proxies, ratproxy, security, testing, xss

Apache proxy auto-re-loader. Neat trick: set your 502 (Bad Gateway) error document to include a meta refresh tag, automating the refresh needed should a server you are proxying to be temporarily unavailable.

# 18th February 2008, 9:44 am / apache, metarefresh, ned-batchelder, proxies

2007

A Fair Proxy Balancer for Nginx and Mongrel. nginx uses round robin for proxying by default; this extension module ensures requests are queued up and sent through to backend mongrel servers that aren’t currently busy. I don’t see any reason this wouldn’t work with servers other than mongrel.

# 9th December 2007, 2:57 pm / fair, load-balancing, mongrel, nginx, proxies

The State of Proxy Caching. If you’ve always wondered exactly what intermediate proxies are going to do to your carefully constructed Web application, here’s your answer.

# 21st June 2007, 2:18 pm / caching, http, mark-nottingham, proxies

Online and offline development with the YUI and Charles (via) Stuart Colville shows how the Charles debugging proxy can be used to serve up hosted YUI files while developing offline.

# 15th May 2007, 2:41 pm / charles, debugging, javascript, offline, proxies, stuart-colville, yui