Simon Willison’s Weblog

Your Web, Half a Second Sooner. Google AdSense now serves a tiny bit of JavaScript that loads everything else in a dynamically populated iframe, thus avoiding blocking the rest of the page load. It’s about time online advertising providers started taking page performance seriously. 17th March 2011, 5:39 pm

Google and Microsoft Cheat on Slow-Start. Should You? Fascinating optimisation tricks by some of the big websites, which violate the RFC governing the TCP slow-start algorithm in order to perform better in the common case. 3rd December 2010, 7:03 pm

Bees with machine guns! Low-cost, distributed load-testing using EC2. Great name for a useful project—Bees with machine guns is a Fabric script which fires up a bunch of EC2 instances, uses them to load test a website and then spins them back down again. 27th October 2010, 11:04 pm

Velocity: Forcing Gzip Compression. Almost every browser supports gzip these days, but 15% of web requests have had their Accept-Encoding header stripped or mangled, generally due to poorly implemented proxies or anti-virus software. Steve Souders passes on a trick used by Google Search, where an iframe is used to test the browser’s gzip support and set a cookie to force gzipping of future pages. 30th September 2010, 5:45 pm

Diffable: only download the deltas. JavaScript library for detecting and serving diffs to JavaScript rather than downloading large scripts every time a few lines of code are changed. “Using Diffable has reduced page load times in Google Maps by more than 1200 milliseconds (~25%). Note that this benefit only affects users that have an older version of the script in cache. For Google Maps that’s 20-25% of users.” 11th July 2010, 12:19 pm

Lazy Load Plugin for jQuery. I’m using this jQuery plugin to save some bandwidth when people first view my Redis tutorial slides. It unobtrusively replaces images on a page with a placeholder graphic, then sets them to load automatically as the user scrolls down the page. 26th April 2010, 12:02 am

Introducing the PyPy 1.2 release. It’s been a long time coming, but 1.2 is the first PyPy release to ship with a Just-in-Time compiler! Performance looks pretty impressive. 12th March 2010, 11:54 pm

Is johnny-cache for you? “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. 2nd March 2010, 11:44 am

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. 28th February 2010, 10:55 pm

Making Facebook 2x Faster. Facebook have a system called BigPipe which allows them to progressively send their pages to the browser as the server-side processing completes to optimise client loading time. Anyone reverse engineered this yet to figure out how they actually do it? 19th February 2010, 9:14 am

HipHop for PHP: Move Fast. Facebook have open-sourced their internally developed PHP to C++ compiler. They serve 400 billion PHP pages a month (that’s more than 150,000 a second) so any performance improvement dramatically reduces their hardware costs, and HipHop drops the CPU usage on their web servers by an average of 50%. “We are serving over 90% of our Web traffic using HipHop, all only six months after deployment”. 2nd February 2010, 6:59 pm

Dojo 1.4.1 vs jQuery 1.4.2pre on Taskspeed. John Resig’s reponse. When JavaScript libraries compete on performance, everybody wins. 29th January 2010, 2:19 pm

Dojo: Still Twice As Fast When It Matters Most. Alex Russell shows how Dojo out-performs jQuery on the TaskSpeed benchmark, which attempts to represent common tasks in real-world applications and has had code that have been optimised by the development teams behind each of the libraries. 28th January 2010, 10:40 pm

Reexamining Python 3 Text I/O. Python 3.1’s IO performance is a huge improvement over 3.0, but still considerably slower than 2.6. It turns out it’s all to do with Python 3’s unicode support: When you read a file in to a string, you’re asking Python to decode the bytes in to UTF-8 (the new default encoding) at the same time. If you open the file in binary mode Python 3 will read raw bytes in to a bytestring instead, avoiding the conversion overhead and performing only 4% slower than the equivalent code in Python 2.6.4. 28th January 2010, 1:28 pm

Linux performance basics. This kind of Linux knowledge is rapidly becoming a key skill for server-side web development. 24th January 2010, 1:50 pm

HTTP + Politics = ? Mark Nottingham ponders the technical implications of Australia’s decision to apply a filter to all internet traffic. Australia is large enough (and far enough away from the northern hemisphere) that the speed of light is a performance issue, but filtering technologies play extremely poorly with optimisation technologies such as HTTP pipelining and Google’s SPDY proposal. 15th December 2009, 3:36 pm

jQuery 1.4 Alpha 1 Released. Impressively the new version contains no new features at all (correct me if I’m wrong), instead focusing on significant performance improvements to the existing API. 5th December 2009, 5:31 pm

Google Analytics goes async. This is excellent news—the latest version of the Google Analytics JavaScript is designed to allow for asynchronous loading, so it won’t hold up the rendering of your page. Analytics and banner ads are the two worst offenders when it comes to slowing down page loads. Now if only a banner ad vendor would follow suit... 2nd December 2009, 6:30 pm

LABjs: new hotness for script loading. Created in collaboration with Steve Souders, LABjs is a JavaScript loading library which makes it easy to have scripts download in parallel while still ensuring that they execute sequentially where required to ensure dependencies are met. It’s unclear how you would decide to use this over concatenating all scripts together in to a single file. 26th November 2009, 12:28 pm

UK Scale Camp. We’re hosting a one day web performance and scalability unconference at the Guardian on the 4th of December. If you’re involved in running a high-scale website in the UK (or abroad) we’d love you to come along. Spaces are going fast. 4th November 2009, 11:12 pm

Using Graphics Card Memory as Swap (via) Interesting idea: “Graphic cards contain a lot of very fast RAM, typically between 64 and 512 MB. With Linux, it’s possible to use it as swap space, or even as RAM disk.” 3rd November 2009, 11:01 am

High-end Varnish-tuning. Tuning the Varnish HTTP cache to serve 27K requests/second on a single core 2.2GHz Opteron. 20th October 2009, 9:25 am

The State of Solid State Hard Drives. From Jeff Atwood’s report it sounds like the price/performance ratio for SSD hard drives has got to a point where switching is the most cost effective way of improving a personal machine’s performance. Anyone know what’s involved in putting one of these things in a MacBook Pro? 14th October 2009, 1:03 pm

Gmail for Mobile: Reducing Startup Latency. Cheeky iPhone optimisation trick—parsing 200 KB of JavaScript takes an iPhone 2.2 device 2.6 seconds, so Gmail embeds code components in /* comments */ in a script tag and evals them on demand later on when the features are needed. 23rd September 2009, 10:29 pm

Memcached 1.4.0 released. The big new feature is the (optional) binary protocol, which enables other features such as CAS-everywhere and efficient client-side replication. Maintainer Dustin Sallings has also released some useful sounding EC2 instances which automatically assign nearly all of their RAM to memcached on launch and shouldn’t need any further configuration. 17th July 2009, 10:26 pm

Firefox 3.5 for developers. It’s out today, and the feature list is huge. Highlights include HTML 5 drag ’n’ drop, audio and video elements, offline resources, downloadable fonts, text-shadow, CSS transforms with -moz-transform, localStorage, geolocation, web workers, trackpad swipe events, native JSON, cross-site HTTP requests, text API for canvas, defer attribute for the script element and TraceMonkey for better JS performance! 30th June 2009, 6:08 pm

What’s New In Python 3.1. Lots of stuff, but the best bits are an ordered dictionary type (congrats, Armin), a Counter class for counting unique items in an iterable (I do this on an almost daily basis) and a bunch of performance improvements including a rewrite of the Python 3.0 IO system in C. 28th June 2009, 3:02 pm

To Sprite Or Not To Sprite. CSS sprite images are decompressed to full bitmaps by browsers before they are rendered, so sprite files with large numbers of pixels will dramatically increase the memory footprint of your site. 24th June 2009, 10:33 am

Django tip: Caching and two-phased template rendering. Neat trick for expensive pages which can be mostly cached with the exception of the “logged in as” bit—run them through the template system twice, caching the intermediary generated template. 19th May 2009, 1:34 am

Loading Scripts Without Blocking. Steve Souders is publishing extracts from his new book, “Even Faster Web Sites”. Here’s a systematic study of different JavaScript loading methods, along with a decision tree for picking the most appropriate one for your application. 30th April 2009, 7:56 pm

peeping into memcached. “Peep uses ptrace to freeze a running memcached server, dump the internal key metadata, and return the server to a running state”—you can then load the resulting data in to MySQL using LOAD LOCAL INFILE and analyse it using standard SQL queries. 20th April 2009, 6:35 pm

ProjectPlan—unladen-swallow. A branch of Python 2.6 aiming to radically improve performance (the target is a 5x improvement), by compiling Python to machine code using LLVM’s JIT engine. I think this is a Google 20% time project (or maybe not, see the comments). An early version without LLVM is already available for download. 30th March 2009, 10:09 am

Magic properties make Firefox synchronously load the Java plugin. Even defining a function called sun() (or several other symbols) will trigger the Java VM to be loaded, dramatically hurting the performance of your page. 27th February 2009, 4:03 pm

jQuery 1.3.1 Released. Bug fix for 1.3, mainly browser compatibility issues. Of interest: jQuery no longer ship a packed version (where JS is used to further decompress a string), as their tests show that this reduces performance due to the overhead of the extra decompression. They still provide a YUI Compressor minified version. 22nd January 2009, 10:41 am

Sloppy—the slow proxy. Java Web Start GUI application which runs a proxy to the site of your choice simulating lower connection speeds—great for testing how well your ajax holds up under poor network conditions. 13th January 2009, 4:17 pm

simplejson 2.0.1. Python’s simplejson JSON library got a whole lot faster while I wasn’t looking. 1st October 2008, 10:55 pm

Dromaeo: JavaScript Performance Testing (via) This is one classy benchmark. Run it in as many browsers as you like (each run is saved to the server and assigned a run ID), then compare the results by appending ?id=[run1],[run2]... to the URL. 11th September 2008, 4:06 pm

TraceMonkey. Brendan Eich has been preaching the performance benefits of tracing and JIT for JavaScript on the conference circuit for at least a year, and the results from the first effort to be merged in to Mozilla core are indeed pretty astounding. 22nd August 2008, 11:13 pm

ZeroMQ. Open source message queue optimised for performance: claims 25μsec latency and 2.6 million messages per second. 27th July 2008, 4:57 pm

SquirrelFish. WebKit’s JavaScript engine was no slouch, but that hasn’t stopped them from replacing it with a brand new “register-based, direct-threaded, high-level bytecode engine, with a sliding register window calling convention”. It runs 1.6x faster and has the Best Logo Ever. 3rd June 2008, 7:57 am

Yahoo!’s Latest Performance Breakthroughs. 20 new performance tips to join the previously published 14. Flushing the buffer while the backend code is still working to cause the browser to start loading CSS earlier is interesting. 20th March 2008, 3:17 pm

IE8 speeds things up. Steve Souders notes that IE8 downloads script files in parallel before executing them sequentially, giving it a significant speed boost over other browsers that download sequentially. 11th March 2008, 5:42 am

Nginx and Memcached, a 400% boost! Ilya Grigorik wrote up my current favourite nginx trick—you set nginx to check memcached for a cache entry matching the current URL on every hit, then invalidate your cache by pushing a new cache record straight in to memcached from your application server. 11th February 2008, 10:05 pm

Announcing StaticGenerator for Django. Simple but powerful static file generator for Django applications—just tell it about your model instances and it will create an entire static site based on calling get_absolute_url() on each one. Uses signals to repopulate the cache when a model changes. 7th January 2008, 9:26 pm

20,000 Reasons Why Comet Scales. Greg Wilkins coaxes Jetty and Bayeux in to supporting 20,000 simultaneous users per server while maintaining sub-second latency, using Amazon EC2 to run the benchmark. 7th January 2008, 8:32 am

Ruby 1.9—Right for You? Dave Thomas on the just-released Ruby 1.9. It’s a development release that breaks backwards compatibility in a few minor ways, but new features include the YARV virtual machine (hence significant speed improvements) and unicode support via associating encodings with bytestrings. 26th December 2007, 12:09 pm

ErlyWeb vs. Ruby on Rails EC2 Performance Showdown. ErlyWeb’s peak response rate beats Rails by 47x, albeit with a hugely simplified benchmark. More interesting than the results is the idea of using EC2 for benchmarking on identical simulated hardware. 10th December 2007, 3:27 pm

Yahoo! Search Contextual Precaching. Neat performance trick on Yahoo! Search: the moment you start typing (indicating you intend to search) the site quietly fires off a bunch of requests to precache assets needed for the search results page. 16th November 2007, 3:58 pm

CSS Sprite Generator (via) Upload a zip file of images and get back a CSS sprite plus a set of pre-calculated background image rules. Tool built by Ed Eliot and Stuart Colville for their forthcoming book “High Performance Web Site Techniques”. 27th September 2007, 10:59 pm

YSlow. New extension for Firebug (yes, an extension on top of another extension) from the Yahoo! performance team which provides improved performance measurement tools and optimisation advice. 25th July 2007, 4:48 am

Return of the HTTP overhead delay. Christian proposes a neat way of improving page performance, by delaying non-essential images such as avatars until after the rest of the page has loaded. 11th July 2007, 3:12 pm

Reducing HTTP requests using make. Nice simple recipe for concatenating JavaScript in to one file using make—doesn’t do anything for cache-busting though. 27th May 2007, 11:29 pm

Rapid development serving 500,000 pages/hour (via) Curse Gaming are getting impressive performance out of Django. 24th May 2007, 4:11 pm

Content delivery system design mistakes. Collection of tips for optimising Web server performance. Mentions lighttpd/nginx, Keep-Alive, expires headers, noatime and more. 15th February 2007, 11 am

Browser Cache Usage—Exposed! Includes real numbers for browser cache usage on some of Yahoo!’s most popular pages. 7th January 2007, 10:20 pm