Simon Willison’s Weblog

Is johnny-cache for you? “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. # 2nd March 2010, 11:44 am

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. # 28th February 2010, 10:55 pm

Making Facebook 2x Faster. Facebook have a system called BigPipe which allows them to progressively send their pages to the browser as the server-side processing completes to optimise client loading time. Anyone reverse engineered this yet to figure out how they actually do it? # 19th February 2010, 9:14 am

HipHop for PHP: Move Fast. Facebook have open-sourced their internally developed PHP to C++ compiler. They serve 400 billion PHP pages a month (that’s more than 150,000 a second) so any performance improvement dramatically reduces their hardware costs, and HipHop drops the CPU usage on their web servers by an average of 50%. “We are serving over 90% of our Web traffic using HipHop, all only six months after deployment”. # 2nd February 2010, 6:59 pm

Dojo 1.4.1 vs jQuery 1.4.2pre on Taskspeed. John Resig’s reponse. When JavaScript libraries compete on performance, everybody wins. # 29th January 2010, 2:19 pm

Dojo: Still Twice As Fast When It Matters Most. Alex Russell shows how Dojo out-performs jQuery on the TaskSpeed benchmark, which attempts to represent common tasks in real-world applications and has had code that have been optimised by the development teams behind each of the libraries. # 28th January 2010, 10:40 pm

Reexamining Python 3 Text I/O. Python 3.1’s IO performance is a huge improvement over 3.0, but still considerably slower than 2.6. It turns out it’s all to do with Python 3’s unicode support: When you read a file in to a string, you’re asking Python to decode the bytes in to UTF-8 (the new default encoding) at the same time. If you open the file in binary mode Python 3 will read raw bytes in to a bytestring instead, avoiding the conversion overhead and performing only 4% slower than the equivalent code in Python 2.6.4. # 28th January 2010, 1:28 pm

Linux performance basics. This kind of Linux knowledge is rapidly becoming a key skill for server-side web development. # 24th January 2010, 1:50 pm

HTTP + Politics = ? Mark Nottingham ponders the technical implications of Australia’s decision to apply a filter to all internet traffic. Australia is large enough (and far enough away from the northern hemisphere) that the speed of light is a performance issue, but filtering technologies play extremely poorly with optimisation technologies such as HTTP pipelining and Google’s SPDY proposal. # 15th December 2009, 3:36 pm

jQuery 1.4 Alpha 1 Released. Impressively the new version contains no new features at all (correct me if I’m wrong), instead focusing on significant performance improvements to the existing API. # 5th December 2009, 5:31 pm

Google Analytics goes async. This is excellent news—the latest version of the Google Analytics JavaScript is designed to allow for asynchronous loading, so it won’t hold up the rendering of your page. Analytics and banner ads are the two worst offenders when it comes to slowing down page loads. Now if only a banner ad vendor would follow suit... # 2nd December 2009, 6:30 pm

LABjs: new hotness for script loading. Created in collaboration with Steve Souders, LABjs is a JavaScript loading library which makes it easy to have scripts download in parallel while still ensuring that they execute sequentially where required to ensure dependencies are met. It’s unclear how you would decide to use this over concatenating all scripts together in to a single file. # 26th November 2009, 12:28 pm

UK Scale Camp. We’re hosting a one day web performance and scalability unconference at the Guardian on the 4th of December. If you’re involved in running a high-scale website in the UK (or abroad) we’d love you to come along. Spaces are going fast. # 4th November 2009, 11:12 pm

Using Graphics Card Memory as Swap (via) Interesting idea: “Graphic cards contain a lot of very fast RAM, typically between 64 and 512 MB. With Linux, it’s possible to use it as swap space, or even as RAM disk.” # 3rd November 2009, 11:01 am

High-end Varnish-tuning. Tuning the Varnish HTTP cache to serve 27K requests/second on a single core 2.2GHz Opteron. # 20th October 2009, 9:25 am

The State of Solid State Hard Drives. From Jeff Atwood’s report it sounds like the price/performance ratio for SSD hard drives has got to a point where switching is the most cost effective way of improving a personal machine’s performance. Anyone know what’s involved in putting one of these things in a MacBook Pro? # 14th October 2009, 1:03 pm

Gmail for Mobile: Reducing Startup Latency. Cheeky iPhone optimisation trick—parsing 200 KB of JavaScript takes an iPhone 2.2 device 2.6 seconds, so Gmail embeds code components in /* comments */ in a script tag and evals them on demand later on when the features are needed. # 23rd September 2009, 10:29 pm

Memcached 1.4.0 released. The big new feature is the (optional) binary protocol, which enables other features such as CAS-everywhere and efficient client-side replication. Maintainer Dustin Sallings has also released some useful sounding EC2 instances which automatically assign nearly all of their RAM to memcached on launch and shouldn’t need any further configuration. # 17th July 2009, 10:26 pm

Firefox 3.5 for developers. It’s out today, and the feature list is huge. Highlights include HTML 5 drag ’n’ drop, audio and video elements, offline resources, downloadable fonts, text-shadow, CSS transforms with -moz-transform, localStorage, geolocation, web workers, trackpad swipe events, native JSON, cross-site HTTP requests, text API for canvas, defer attribute for the script element and TraceMonkey for better JS performance! # 30th June 2009, 6:08 pm

What’s New In Python 3.1. Lots of stuff, but the best bits are an ordered dictionary type (congrats, Armin), a Counter class for counting unique items in an iterable (I do this on an almost daily basis) and a bunch of performance improvements including a rewrite of the Python 3.0 IO system in C. # 28th June 2009, 3:02 pm

To Sprite Or Not To Sprite. CSS sprite images are decompressed to full bitmaps by browsers before they are rendered, so sprite files with large numbers of pixels will dramatically increase the memory footprint of your site. # 24th June 2009, 10:33 am

Django tip: Caching and two-phased template rendering. Neat trick for expensive pages which can be mostly cached with the exception of the “logged in as” bit—run them through the template system twice, caching the intermediary generated template. # 19th May 2009, 1:34 am

Loading Scripts Without Blocking. Steve Souders is publishing extracts from his new book, “Even Faster Web Sites”. Here’s a systematic study of different JavaScript loading methods, along with a decision tree for picking the most appropriate one for your application. # 30th April 2009, 7:56 pm

peeping into memcached. “Peep uses ptrace to freeze a running memcached server, dump the internal key metadata, and return the server to a running state”—you can then load the resulting data in to MySQL using LOAD LOCAL INFILE and analyse it using standard SQL queries. # 20th April 2009, 6:35 pm

ProjectPlan—unladen-swallow. A branch of Python 2.6 aiming to radically improve performance (the target is a 5x improvement), by compiling Python to machine code using LLVM’s JIT engine. I think this is a Google 20% time project (or maybe not, see the comments). An early version without LLVM is already available for download. # 30th March 2009, 10:09 am

Magic properties make Firefox synchronously load the Java plugin. Even defining a function called sun() (or several other symbols) will trigger the Java VM to be loaded, dramatically hurting the performance of your page. # 27th February 2009, 4:03 pm

jQuery 1.3.1 Released. Bug fix for 1.3, mainly browser compatibility issues. Of interest: jQuery no longer ship a packed version (where JS is used to further decompress a string), as their tests show that this reduces performance due to the overhead of the extra decompression. They still provide a YUI Compressor minified version. # 22nd January 2009, 10:41 am

Sloppy—the slow proxy. Java Web Start GUI application which runs a proxy to the site of your choice simulating lower connection speeds—great for testing how well your ajax holds up under poor network conditions. # 13th January 2009, 4:17 pm