Simon Willison’s Weblog

66 items tagged “performance”

2021

Cleaning Up Your Postgres Database (via) Craig Kerstiens provides some invaluable tips on running an initial check of the health of a PostgreSQL database, by using queries against the pg_statio_user_indexes table to find the memory cache hit ratio and the pg_stat_user_tables table to see what percentage of queries to your tables are using an index. # 3rd February 2021, 7:32 am

Making GitHub’s new homepage fast and performant. A couple of really clever tricks in this article by Tobias Ahlin. The first is using IntersectionObserver in conjunction with the video preload=“none” attribute to lazily load a video when it scrolls into view. The second is an ingenious trick to create an efficiently encoded transparent JPEG image: embed the image in a SVG file twice, once as the image and once as a transparency mask. # 29th January 2021, 7:05 pm

2020

How Shopify Uses WebAssembly Outside of the Browser (via) I’m fascinated by applications of WebAssembly outside the browser. As a Python programmer I’m excited to see native code libraries getting compiled to WASM in a way that lets me call them from Python code via a bridge, but the other interesting application is executing untrusted code in a sandbox. Shopify are doing exactly that—they are building a kind-of plugin mechanism where partner code compiled to WASM runs inside their architecture using Fastly’s Lucet. The performance numbers are in the same ballpark as native code. Also interesting: they’re recommending AssemblyScript, a TypeScript-style language designed to compile directly to WASM without needing any additional interpreter support, as required by dynamic languages such as JavaScript, Python or Ruby. # 19th December 2020, 4:46 pm

I’ve really come to appreciate that performance isn’t just some property of a tool independent from its functionality or its feature set. Performance — in particular, being notably fast — is a feature in and of its own right, which fundamentally alters how a tool is used and perceived.

Nelson Elhage # 24th February 2020, 2:32 pm

2018

The Now CDN (via) Huge announcement from Zeit Now today: all .now.sh deployments are now served through the Cloudflare CDN, which means they benefit from 150 worldwide CDN locations that obey HTTP caching headers. This is particularly relevant for Datasette, since it serves far-future cache headers by default and uses Cloudflare-compatible HTTP/2 push hints to accelerate 302 redirects. This means that both the “datasette publish now” CLI command and the Datasette Publish web app will now result in Cloudflare-accelerated deployments. # 12th July 2018, 3:34 am

The latest SQLite 3.8.7 alpha version is 50% faster than the 3.7.17 release from 16 months ago.  That is to say, it does 50% more work using the same number of CPU cycles. [...] The 50% faster number above is not about better query plans.  This is 50% faster at the low-level grunt work of moving bits on and off disk and search b-trees.  We have achieved this by incorporating hundreds of micro-optimizations.  Each micro-optimization might improve the performance by as little as 0.05%.  If we get one that improves performance by 0.25%, that is considered a huge win.  Each of these optimizations is unmeasurable on a real-world system (we have to use cachegrind to get repeatable run-times) but if you do enough of them, they add up.

D. Richard Hipp # 10th May 2018, 5:15 am

2017

Computer latency: 1977-2017 (via) Dan Luu used a 240 fps camera to investigate the latency between hitting a key and having the character show up on the display across four decades of computing devices... and found 1983’s Apple IIe outperformed everything else. He goes to great lengths to explain why in his accompanying write-up. # 26th December 2017, 12:28 am

Many Small Queries Are Efficient In SQLite. Since SQLite runs in-process rather than being accessed over a network it avoids the per-query overhead of network round trips. This means that while MySQL or PostgreSQL applications need to avoid N+1 query patterns that create 100s of queries per request, SQLite apps can be designed differently: provided you hit indexes or small tables, 200 queries just means 200 extra cheap function calls. # 26th November 2017, 4:24 pm

2011

Your Web, Half a Second Sooner. Google AdSense now serves a tiny bit of JavaScript that loads everything else in a dynamically populated iframe, thus avoiding blocking the rest of the page load. It’s about time online advertising providers started taking page performance seriously. # 17th March 2011, 5:39 pm

2010

Google and Microsoft Cheat on Slow-Start. Should You? Fascinating optimisation tricks by some of the big websites, which violate the RFC governing the TCP slow-start algorithm in order to perform better in the common case. # 3rd December 2010, 7:03 pm

Bees with machine guns! Low-cost, distributed load-testing using EC2. Great name for a useful project—Bees with machine guns is a Fabric script which fires up a bunch of EC2 instances, uses them to load test a website and then spins them back down again. # 27th October 2010, 11:04 pm

Velocity: Forcing Gzip Compression. Almost every browser supports gzip these days, but 15% of web requests have had their Accept-Encoding header stripped or mangled, generally due to poorly implemented proxies or anti-virus software. Steve Souders passes on a trick used by Google Search, where an iframe is used to test the browser’s gzip support and set a cookie to force gzipping of future pages. # 30th September 2010, 5:45 pm

Diffable: only download the deltas. JavaScript library for detecting and serving diffs to JavaScript rather than downloading large scripts every time a few lines of code are changed. “Using Diffable has reduced page load times in Google Maps by more than 1200 milliseconds (~25%). Note that this benefit only affects users that have an older version of the script in cache. For Google Maps that’s 20-25% of users.” # 11th July 2010, 12:19 pm

Lazy Load Plugin for jQuery. I’m using this jQuery plugin to save some bandwidth when people first view my Redis tutorial slides. It unobtrusively replaces images on a page with a placeholder graphic, then sets them to load automatically as the user scrolls down the page. # 26th April 2010, 12:02 am

Introducing the PyPy 1.2 release. It’s been a long time coming, but 1.2 is the first PyPy release to ship with a Just-in-Time compiler! Performance looks pretty impressive. # 12th March 2010, 11:54 pm

Is johnny-cache for you? “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. # 2nd March 2010, 11:44 am

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. # 28th February 2010, 10:55 pm

Making Facebook 2x Faster. Facebook have a system called BigPipe which allows them to progressively send their pages to the browser as the server-side processing completes to optimise client loading time. Anyone reverse engineered this yet to figure out how they actually do it? # 19th February 2010, 9:14 am

HipHop for PHP: Move Fast. Facebook have open-sourced their internally developed PHP to C++ compiler. They serve 400 billion PHP pages a month (that’s more than 150,000 a second) so any performance improvement dramatically reduces their hardware costs, and HipHop drops the CPU usage on their web servers by an average of 50%. “We are serving over 90% of our Web traffic using HipHop, all only six months after deployment”. # 2nd February 2010, 6:59 pm

Dojo 1.4.1 vs jQuery 1.4.2pre on Taskspeed. John Resig’s reponse. When JavaScript libraries compete on performance, everybody wins. # 29th January 2010, 2:19 pm

Dojo: Still Twice As Fast When It Matters Most. Alex Russell shows how Dojo out-performs jQuery on the TaskSpeed benchmark, which attempts to represent common tasks in real-world applications and has had code that have been optimised by the development teams behind each of the libraries. # 28th January 2010, 10:40 pm

Reexamining Python 3 Text I/O. Python 3.1’s IO performance is a huge improvement over 3.0, but still considerably slower than 2.6. It turns out it’s all to do with Python 3’s unicode support: When you read a file in to a string, you’re asking Python to decode the bytes in to UTF-8 (the new default encoding) at the same time. If you open the file in binary mode Python 3 will read raw bytes in to a bytestring instead, avoiding the conversion overhead and performing only 4% slower than the equivalent code in Python 2.6.4. # 28th January 2010, 1:28 pm

Linux performance basics. This kind of Linux knowledge is rapidly becoming a key skill for server-side web development. # 24th January 2010, 1:50 pm

2009

HTTP + Politics = ? Mark Nottingham ponders the technical implications of Australia’s decision to apply a filter to all internet traffic. Australia is large enough (and far enough away from the northern hemisphere) that the speed of light is a performance issue, but filtering technologies play extremely poorly with optimisation technologies such as HTTP pipelining and Google’s SPDY proposal. # 15th December 2009, 3:36 pm

jQuery 1.4 Alpha 1 Released. Impressively the new version contains no new features at all (correct me if I’m wrong), instead focusing on significant performance improvements to the existing API. # 5th December 2009, 5:31 pm

Google Analytics goes async. This is excellent news—the latest version of the Google Analytics JavaScript is designed to allow for asynchronous loading, so it won’t hold up the rendering of your page. Analytics and banner ads are the two worst offenders when it comes to slowing down page loads. Now if only a banner ad vendor would follow suit... # 2nd December 2009, 6:30 pm

LABjs: new hotness for script loading. Created in collaboration with Steve Souders, LABjs is a JavaScript loading library which makes it easy to have scripts download in parallel while still ensuring that they execute sequentially where required to ensure dependencies are met. It’s unclear how you would decide to use this over concatenating all scripts together in to a single file. # 26th November 2009, 12:28 pm

UK Scale Camp. We’re hosting a one day web performance and scalability unconference at the Guardian on the 4th of December. If you’re involved in running a high-scale website in the UK (or abroad) we’d love you to come along. Spaces are going fast. # 4th November 2009, 11:12 pm

Using Graphics Card Memory as Swap (via) Interesting idea: “Graphic cards contain a lot of very fast RAM, typically between 64 and 512 MB. With Linux, it’s possible to use it as swap space, or even as RAM disk.” # 3rd November 2009, 11:01 am

Why I like Redis

I’ve been getting a lot of useful work done with Redis recently.

[... 900 words]