Blogmarks
Filters: Sorted by date
Introducing BERT and BERT-RPC. Justification for inventing a brand new serialisation protocol: Thrift and Protocol Buffers both use IDLs and code generation, XML “is not convertible to a simple unambiguous data structure in any language I’ve ever used” and JSON lacks support for unencoded binary data. The result is BERT—Binary ERlang Term—which extracts a format from Erlang in much the same way that JSON extracted one from JavaScript.
How We Made GitHub Fast. Detailed overview of the new GitHub architecture. It’s a lot more complicated than I would have expected—lots of moving parts are involved in ensuring they can scale horizontally when they need to. Interesting components include nginx, Unicorn, Rails, DRBD, HAProxy, Redis, Erlang, memcached, SSH, git and a bunch of interesting new open source projects produced by the GitHub team such as BERT/Ernie and ProxyMachine.
Introducing Cloudera Desktop. It’s a GUI for Hadoop, and under the hood is a whole stack of open source software, including Python, Django, MooTools, Twisted, lxml, CherryPy, Mako, Java and AspectJ.
Comcast: Twitter Has Changed The Culture Of Our Company. “Frank Eliason (@Comcastcares on Twitter) now has 11 people working under him simply to respond to information about Comcast being broadcast on Twitter.”
High-end Varnish-tuning. Tuning the Varnish HTTP cache to serve 27K requests/second on a single core 2.2GHz Opteron.
nginx_http_push_module. More clever design with webhooks—here’s an nginx module that provides a comet endpoint URL which will hang until a back end process POSTs to another URL on the same server. This makes it much easier to build asynchronous comet apps using regular synchronous web frameworks such as Django, PHP and Rails.
“I made the first animated under construction icon”. twoleftfeet on MetaFilter describes how he created the first ever Under Construction animation in 1995, after discovering his server-push animations could be replaced by the exciting new animated GIF.
The State of Solid State Hard Drives. From Jeff Atwood’s report it sounds like the price/performance ratio for SSD hard drives has got to a point where switching is the most cost effective way of improving a personal machine’s performance. Anyone know what’s involved in putting one of these things in a MacBook Pro?
Temporary Mapping: Solar Decathlon. The OpenStreetMap default renderer supports start_date and end_date tags, meaning you can map temporary installations (in this case the 2009 Solar Decathlon on the DC National Mall) and have them automatically appear and disappear at the correct times.
MySQL backups with EBS snapshots. Assaf Arkin’s 45 line ruby script shows how to lock tables / XFS freeze / create an EBS snapshot / unfreeze and unlock, with hourly snapshots preserved for the past 24 hours and daily snapshots for the past week. Is an EBS snapshot enough to restore your data to somewhere other than EC2 though?
OSM static map api. A very welcome addition to the OpenStreetMap world (with plenty of options for overlaying points, polygons etc) slightly marred by the size and relative ugliness of the OpenStreetMap watermark.
OpenStreetMap Rendering Database. Amazon have added an OpenStreetMap snapshot as a public data set, thanks to some smart prompting by Jeremy Dunck.
WebKit, Mobile, and Progress. Alex Russell responds to PPK’s analysis of the many different WebKit variants in today’s mobile phones, pointing out that the replacement cycle and increasing quality of WebKit in more recent phones means the situation still looks pretty good.
Django security updates released. A potential denial of service vulnerability has been discovered in the regular expressions used by Django form library’s EmailField and URLField—a malicious input could trigger a pathological performance. Patches (and patched releases) for Django 1.1 and Django 1.0 have been published.
Micro Men. “Affectionately comic drama about the British home computer boom of the early 1980s.”—aired last night, and on BBC iPlayer for the next week. I thought it was absolutely charming, as well as being a thought provoking history of the rise and fall of the British computer industry in the early 80s.
MichaelMoore.com in Django. A seriously impressive case study—a complete rebuild from the ground up completed in just five weeks using Django, Solr and Haystack for a high traffic site with a top 10,000 US Alexa ranking.
Cloudvox. A brand new startup offering “API-driven phone calls” with a beautifully simple webhooks based API.
Official Google Webmaster Blog: A proposal for making AJAX crawlable.
It's horrible! The Google crawler would map url#!state to url?_escaped_fragment_=state, then expect your site to provide rendered HTML that reflects that state (they even go as far as to suggest running a headless browser within your web server to do this). Just stick to progressive enhancement instead, it's far less hideous. It looks like the proposal may have originated with the GWT team.
XSS Protection by Default in Rails 3.0. Fantastic news—congratulations, Rails core team.
Twisted Web in 60 seconds. A common complaint about Twisted is how hard it is to figure out the web stack. Jp Calderon’s tutorial (in nine installments and counting) is the best documentation on web development in Twisted I’ve seen.
breaking links. Mike complains about sites such as Twitter and WordPress.com which mess around with Ajax and links and hence breaks the ability to command-click to open a new tab in Safari (and Chrome). I just realised that I’ve subconsciously retrained myself to right click and select “open in new tab” to avoid that exact issue.
There is no WebKit on Mobile. PPK ran 27 tests against 19 different WebKit-on-mobile implementations and found enormous disparities between the levels of support in currently available mobile phones.
Python is Unix. Jacob ports Ryan Tomayko’s simple prefork network server to Python.
I like Unicorn because it’s Unix. Ryan Tomayko analyses Unicorn, a new, pre-forking Ruby HTTP server that makes extensive use of Unix syscalls and idioms, and asks why dynamic language programmers don’t take advantage of these more often.
PhotoSketch turns a rough sketch in to a photo montage (via) Computer vision is really exciting at the moment—Photosketch is an application which takes a rough labeled sketch, finds images matching the labels, filters them by the sketched shapes and composes them in to a not-too-bad photo montage. As wmf on Hacker News points out, “this technology has epic potential in the LOLcat market”.
Developing for the Apple iPhone using Flash. A brilliant feat of engineering: Adobe worked around Apple’s “no runtime allowed” rules by writing a compiler front end for LLVM that compiles ActionScript 3 to ARM assembly code, and apparently ported the regular Flash drawing APIs as well.
Royal Mail: closing job search over data dispute while sacking workers. The Royal Mail have cease-and-desisted ernestmarples.com, a free postcode geocoding API which powers a number of UK open democracy sites. One of the sites is Job Centre Pro Plus, creating a perfect opportunity for an ORG press release.
Rupert to Internet: It’s War! Fascinating essay on Rupert Murchoch’s plans to charge for access to his company’s websites, by his biographer Michael Wolff.
History of Django’s popularity. “What sequence of events made Django the most popular Python web framework?”—insightful answers from Alex Martelli and James Bennett.
Logos in Lego Town. “Unlike the railways, there have been a multitude of different airline logos in Lego land – indicating a de-regulated market and open competition.”