Simon Willison’s Weblog

HTTP + Politics = ? Mark Nottingham ponders the technical implications of Australia’s decision to apply a filter to all internet traffic. Australia is large enough (and far enough away from the northern hemisphere) that the speed of light is a performance issue, but filtering technologies play extremely poorly with optimisation technologies such as HTTP pipelining and Google’s SPDY proposal. # 15th December 2009, 3:36 pm

Real time online activity monitor example with node.js and WebSocket. A neat exploration of Node.js—first hooking a “tail -f” process up to an HTTP push stream, then combining that with HTML 5 WebSockets to achieve reliable streaming. # 8th December 2009, 11:07 pm

Language Detection: A Witch’s Brew? The Flickr team make the case for using the Accept-Language header over IP detection to pick a site’s language, with a simple UI for switching languages in case you get it wrong. They’ve been using this for two and a half years without any significant problems. # 5th December 2009, 5:30 pm

SPDY: The Web, Only Faster. Alex Russell explains the benefits of Google’s SPDF proposal (a protocol that upgrades HTTP)—including header compression, multiplexing, the ability to send additional resources such as images and stylesheets down without needing the data:uri hack and Comet support built in to the core assumptions of the protocol. # 13th November 2009, 1 pm

node.js. “Evented I/O for V8 JavaScript”—a JavaScript environment built on top of the super-fast V8 engine which provides event-based IO functionality for building highly concurrent TCP and HTTP servers. The API design is superb—everything is achieved using JavaScript events and callbacks (even regular file IO) and the small standard library ships with comprehensive support for HTTP and DNS. Overall it’s very similar to Twisted and friends, but JavaScript’s anonymous function syntax feels more natural than the Python equivalent. It compiles cleanly on Snow Leopard. Definitely a project to watch. # 9th November 2009, 11:25 pm

Traffic Server. Mark Nottingham explains the release of Traffic Server, a new Apache Incubator open source project donated by Yahoo! using code originally developed at Inktomi around a decade ago. Traffic Server is a HTTP proxy/cache, similar to Squid and Varnish (though Traffic Server acts as both a forward and reverse proxy, whereas Varnish only handles reverse). # 1st November 2009, 12:15 pm

High-end Varnish-tuning. Tuning the Varnish HTTP cache to serve 27K requests/second on a single core 2.2GHz Opteron. # 20th October 2009, 9:25 am

cloud-crowd. New parallel processing worker/job queue system with a strikingly elegant architecture. The central server is an HTTP server that manages job requests, which are farmed out to a number of node HTTP servers which fork off worker processes to do the work. All communication is webhook-style JSON, and the servers are implemented in Sinatra and Thin using a tiny amount of code. The web-based monitoring interface is simply beautiful, using canvas to display graphs showing the system’s overall activity. # 21st September 2009, 11:09 pm

PostBin. Handy debugging tool for webhooks—create a TinyURL-style URL, then see a log of any POST requests made to that address. # 21st September 2009, 11:03 pm

PubSub-over-Webhooks with RabbitHub. RabbitMQ, the Erlang-powered AMQP message queue, is growing an HTTP interface based on webhooks and PubSubHubBub. # 1st July 2009, 8:22 pm

The Resource Expert Droid. Like the HTML Validator but for your server’s HTTP headers—extremely useful. # 25th June 2009, 10:06 am

Styling buttons to look like links. Nat has a neat trick for styling submit buttons to look like regular links—so there’s absolutely no excuse for using a “delete” link when you should be using a POST request. # 10th June 2009, 2:11 am

A rev=“canonical” HTTP Header. Chris Shiflett proposes optionally exposing rev=canonical information in an HTTP header, thus allowing sites to discover shorter URLs using just a HEAD request and removing the need to parse HTML. The pingback specification also uses this shortcut. # 12th April 2009, 12:33 pm

Concurrence. Exciting: a Python framework for “creating massively concurrent network applications” (the tutorial benchmarks a Hello World web server at over 8,000 requests a second). It’s implemented on top of libevent using pyrex, can run on either Stackless Python or Greenlets from the py library and ships with a WSGI server, an HTTP client and a DBAPI 2.0 compliant MySQL driver. # 15th March 2009, 1:28 pm

django-springsteen and Distributed Search. Will Larson’s Django search library currently just talks to Yahoo! BOSS, but is designed to be extensible for other external search services. Interestingly, it uses threads to fire off several HTTP requests in parallel from within the Django view. # 25th February 2009, 10:28 pm

Tokyo Cabinet: Beyond Key-Value Store. Useful overview of Yet Another Scalable Key Value Store. Interesting points: multiple backends (hash table, B-Tree, in memory, on disk), a “table” engine which enables more advanced queries, a network server that supports HTTP, memcached or its own binary protocol and the ability to extend the engine with Lua scripts. # 14th February 2009, 11:17 am

Ehy IE8, I Can Has Some Clickjacking Protection? (via) IE8 has built-in protection against clickjacking, but it’s opt-in (with a custom HTTP header) and IE only. It turns out the usual defence against clickjacking (using framebusting JavaScript) doesn’t work in IE as it can be worked around with a security=“restricted” attribute on an iframe. # 29th January 2009, 1:39 pm

ETags And Modification Times In Django. Part of Malcolm’s series of tutorials on implementing advanced HTTP concepts in Django. # 13th December 2008, 9:49 am

ptth (Reverse HTTP) implementation in a browser using Long Poll COMET. Donovan Preston experiments with the cleverly named idea of ptth, where servers send HTTP requests to clients. # 8th December 2008, 5:22 pm

Response Splitting Risk. Important reminder that you should always ensure strings used in HTTP headers don’t contain newlines. # 19th October 2008, 11:58 pm

Versioning REST Web Services. Peter Williams suggests using a vendor MIME media type in the Accept header to specify a required API version, because embedding the API version in the URL itself leads to a single resource ending up with many different URLs, one for each API version. # 13th October 2008, 12:45 pm

Robust Defenses for Cross-Site Request Forgery [PDF]. Fascinating report which introduces the “login CSRF” attack, where an attacker uses CSRF to log a user in to a site (e.g. PayPal) using the attacker’s credentials, then waits for them to submit sensitive information or bind the account to their credit card. The paper also includes an in-depth study of potential protection measures, including research that shows that 3-11% of HTTP requests to a popular ad network have had their referer header stripped. Around 0.05%-0.10% of requests have custom HTTP headers such as X-Requested-By stripped. # 24th September 2008, 9:40 am

Flickr Developer Blog: API Responses as Feeds (via) Flickr API calls that return a “standard photos response” (e.g. flickr.photos.search and flickr.favorites.getList) can now output eight different feed formats as well, including Atom, RSS flavours, geoatom, geordf and KML. Error codes are returned as X-FlickrErrCode HTTP headers. # 25th August 2008, 10:20 pm

Gears API Blog: Gears 0.4 is here! New features are Geolocation, a Blob API for dealing with arbitrary binary data, onprogress() events for tracking HTTP downloads and uploads (meaning progress indicators) and the built-in Gears dialogs localized to 40 languages. # 22nd August 2008, 10:14 am