Blogmarks
Filters: Sorted by date
Database Sharding at Netlog, with MySQL and PHP. Detailed MySQL sharding case study from Netlog, who serve five billion page requests a month using thousands of shards across more than 80 database servers.
jQuery Sparklines. Delightful Sparklines implementation, using canvas or VML in IE. A neat nod towards unobtrusiveness as well: you can specify your data as comma separated values inside a span, then use a single jQuery method call to convert the span in to a sparkline image.
Magic properties make Firefox synchronously load the Java plugin. Even defining a function called sun() (or several other symbols) will trigger the Java VM to be loaded, dramatically hurting the performance of your page.
How FriendFeed uses MySQL to store schema-less data. The pain of altering/ adding indexes to tables with 250 million rows was killing their ability to try out new features, so they’ve moved to storing pickled Python objects and manually creating the indexes they need as denormalised two column tables. These can be created and dropped much more easily, and are continually populated by an off-line index building process.
Accessibility and Degradation in Cappuccino. Ross Boucher from 280 North responds to Drew McLellan.
Kestrel. Twitter’s Robey Pointer rewrote their Starling message queue in 1500 lines of Scala, adding reliable fetch (where consumers can confirm their receipt of an item) and blocking fetches, which reduce the need for consumers to poll for updates (and hence solve my only beef with the original Starling). I haven’t tried running this on a low spec VPS yet but it looks very promising.
What I’ve Learned from Hacker News. I’m always fascinated by online community war stories.
The Cost of Accessibility. Drew McLellan comments on the seemingly inevitable march towards JavaScript dependent applications, and argues that JavaScript frameworks such as Cappuccino have a duty to integrate accessibility in to their core.
django-springsteen and Distributed Search. Will Larson’s Django search library currently just talks to Yahoo! BOSS, but is designed to be extensible for other external search services. Interestingly, it uses threads to fire off several HTTP requests in parallel from within the Django view.
FAPWS3-0.2 (WSGI server based on libev). Another strong contender for Python’s answer to Mongrel—3500 requests/s for static files, 43 for a simple dynamic (Django powered) pages and 4.8 for a heavy SQL query—all benchmarked with 300 concurrent requests.
Building and Scaling a Startup on Rails: 12 Things We Learned the Hard Way. Lessons learned from Posterous. Some good advice in here, in particular “Memcache later: If you memcache first, you will never feel the pain and never learn how bad your database indexes and Rails queries are”. Also recommends using job queues for offline processing of anything that takes more than 200ms.
Oscars 2009: the interactive results | guardian.co.uk. My latest project for the Guardian, put together on very short notice. Updates live as the results are announced, and allows Twitter users to vote on their favourite for each category by sending a specially formatted message to @guardianfilm—jQuery and Ajax polling against S3 under the hood.
jQuery.Rule (via) jQuery plugin for manipulating stylesheet rules. For me, this is the single most important piece of functionality currently missing from the core jQuery API. The ability to add new CSS rules makes an excellent complement to the .live() method added in jQuery 1.3.
Introducing the Karmic Koala, our mascot for Ubuntu 9.10 (via) Ubuntu 9.10 will have a strong focus on cloud computing, including tools for easily creating EC2 AMIs and Eucalyptus, an open-source system for running an EC2-compatible cloud in your own data centre.
jQuery 1.3.2 release notes. Not just a bug fix—there are a number of subtle behaviour changes, including to the :visible/:hidden selectors and the appendTo/prependTo/*To family of methods. I strongly recommend testing and reviewing those changes before upgrading.
Mapping with Isotype (via) I hadn’t heard of Isotype (International System of Typographic Picture Education), a beautiful pictographic language created in the 1930s. This Isotype-inspired atlas is pretty spectacular.
Map Maker for Developers. Tiles from Google’s Map Maker crowdsourcing effort are now available in the JS and static maps APIs on an opt-in basis. Maybe I’m misunderstanding something here, but Google Map Maker seems like a big step backwards for open geographic data. People donate their mapping efforts to Google, who keep them—unlike OpenStreetMap, where the donated efforts are made available under a Creative Commons license.
The History of Python: Adding Support for User-defined Classes. Guido designed the run-time representation first, and tried to design the syntax to include as few new parsing concepts as possible. The origins of explicit self are also explained.
DB2 support for Django is coming. From IBM, under the Apache 2.0 License. I’m not sure if this makes it hard to bundle it with the rest of Django, which uses the BSD license.
Found in space. The Astrometry bot on Flickr (which detects which part of the night sky is contained within your photo and adds notes to some of the more interesting stars) is the most delightful use of the Flickr API I’ve ever seen. This interview provides some background, including a link to a paper on the “scale and rotation invariant hashing algorithm” that is used to build the index.
Announcing django-viewtools. A really excellent idea—run ./manage.py viewtools --pdb /path/on/site/ to debug a view in your Django project that is raising an error using the Python debugger, or use --profile to run the full request cycle for that URL through the profiler.
CloudMade: A Summary of the Future of Mapping. CloudMade are now offering commercially supported APIs on top of OpenStreetMap, including geocoding, routing and tile access libraries in Python/Ruby/Java and a very neat theming tool that lets you design your own map styles. This is really going to kick innovation around OpenStreetMap up a notch.
Dulwich. A pure Python implementation of the Git file format and protocols. Reinforces my impression that a key to Git’s success is stable, well designed and documented on-disk formats.
“Recover my account” link on the login page. For the record, collecting and verifying e-mail addresses is a VERY good idea, even (especially?) if you accept OpenID. A verified e-mail address is still absolutely the best way to deal with lost passwords or “my OpenID isn’t working”.
Write to a Google Spreadsheet from a Python script. I didn’t know Google Spreadsheets could directly serve dynamic images that automatically update when the underlying data changes.
Web Hooks and the Programmable World of Tomorrow. Tour de force presentation on Web Hooks by Jeff Lindsay. Tons of really good ideas—provided your application isn’t Flickr sized, there’s a good chance you could implement web hooks pretty cheaply and unleash a huge flurry of creativity from your users. GitHub makes a great case study here.
Google App Engine 1.1.9 boosts capacity and compatibility. Niall summarises the recent changes to App Engine. urllib and urllib2 support plus massively increased upload limits and request duration quotas will make it a whole lot easier to deploy serious projects on the platform.
The Django and Ubuntu Intrepid Almanac. Will Larson’s impressively comprehensive guide to configuring and securing an Ubuntu VPS from scratch to run Django, using PostgreSQL and Apache/mod_wsgi behind nginx.
Xapian performance comparision with Whoosh. Whoosh appears to be around four times slower than Xapian for indexing and empty cache searches, but Xapian with a full cache blows Whoosh out of the water (5408 searches/second compared to 26.3). Considering how fast Xapian is, that’s still a pretty impressive result for the pure-Python Whoosh.
Tokyo Cabinet and Tokyo Tyrant Presentation. By Tokyo Cabinet author Mikio Hirabayashi. The third leg of the Tokyo tripod is Tokyo Dystopia, a full-text search engine which is presumably a modern replacement for Mikio’s older hyperestraier engine.