Simon Willison’s Weblog

Applications: the real stars of the data.gov.uk launch. A write-up of the data.gov.uk launch event at the Guardian. I demonstrated the Guardian’s World Government Data search engine and a small data.gov.uk inspired feature on WildlifeNearYou. # 27th January 2010, 12:23 pm

After Three Months, Only 35 Subscriptions for Newsday’s Web Site. Not an entirely representative figure, since it doesn’t include the print and cable subscribers who get access to the website as part of their existing package. # 27th January 2010, 8:16 am

OSM the default map in Haiti. A search and rescue team member in Haiti sends word that digital maps constructed by the OpenStreetMap community are spreading by word of mouth and being loaded on to GPS units on the ground. # 25th January 2010, 9:26 pm

The making of the NYT’s Netflix graphic. A database dump from Netflix, some clever hackery in ArcView GIS, hpricot to scrape Metacritic and a lot of careful thought about the UI for navigating the data. # 25th January 2010, 1:11 pm

Fixing the Google Account problem. 3,000+ words explaining how to open a Google Doc invitation sent to an e-mail address that isn’t associated with your Google account. Worth reading just to get an idea for the enormous complexity involved in running a large scale identity system and designing an interface for managing aliases and multiple profiles. Google haven’t got it right yet—has anyone else? # 25th January 2010, 11:21 am

Help pick the best photos, but watch out, it’s addictive! My favourite WildlifeNearYou feature yet—our new tool asks you to pick the best from two photos, then uses the results to rank all of the photos for each species. It’s surprisingly addictive—we had over 5,000 votes in the first two hours, peaking at 4 or 5 votes a second. The feature seems to be staying nice and speedy thanks to Redis under the hood. Photos in the top three for any given species display a medal on their photo page. # 25th January 2010, 12:36 am

Linux performance basics. This kind of Linux knowledge is rapidly becoming a key skill for server-side web development. # 24th January 2010, 1:50 pm

Amazon S3: Versioning Proposal. The us-west-1 S3 bucket region now optionally supports versioning—once enabled on a bucket, all previous versions of keys will be preserved. # 24th January 2010, 1:38 pm

Don’t Hash Secrets. A well written explanation from 2008 of why you must use hmac instead of raw SHA-1 when hashing against a secret. # 24th January 2010, 1:30 pm

The Seven Deadly Sins of Solr. Useful advice on managing and deploying Solr. # 24th January 2010, 1:30 pm

A successful Git branching model (via) This looks eminently sensible. The master branch is used for production-ready code, and is only updated by merging from either release branches or emergency hotfix branches. A develop branch is used for integration (from feature branches), and is branched to create release branches when a release is nearly ready. It’s all comprehensively documented and comes with some well-designed diagrams. # 20th January 2010, 7:30 pm

Owls, Otters, Monkeys and Lions Near You.com. It’s not just Owls—we also registered ottersnearyou.com, monkeysnearyou.com and lionsnearyou.com. We’ll probably stop there though, or this could turn in to a very expensive marketing gimmick. # 19th January 2010, 2:54 pm

owlsnearyou.com. Nat and I built this over the weekend. It asks for your location, then tells you where your nearest Owl is (using sightings data people have entered on WildlifeNearYou.com). If you’re using Firefox 3.6 or an iPhone it grabs your location using the W3C geolocation API so you don’t have to type anything at all. # 19th January 2010, 2:45 pm

Crayola Crayon Colors Multiply Like Rabits. “In 1903, Crayola had eight colors in its standard package. Today, there are 120”—and here’s a brilliantly designed infographic showing how it happened. # 19th January 2010, 2:44 pm

If You Don’t Date Your Work, It Sucks. I learnt this lesson the hard way, when I realised that I had no idea exactly what year I created my earliest web-facing projects. # 18th January 2010, 5:46 pm

Locations of Ancient Woolworths Stores follow Precise Geometrical Pattern. Excellent send-up of shoddy science reporting. # 18th January 2010, 2:51 pm

WildlifeNearYou: Help identify animals in other people’s photos. The first of a number of crowdsourcing-style features we have planned for WildlifeNearYou—users can now help identify the animals in each other’s photos, and photo owners get a simple queue interface to approve or reject the suggestions. # 15th January 2010, 1:35 am

jQuery 1.4 Released. With comprehensive release notes. Huge performance improvements and a ton of very sensible enhancements to the API—far too many to summarise. # 14th January 2010, 10:37 pm

For the Love of Ponies. Bryan Veloso, the discoverer of the Django Pony, makes contact with the artist who released the original drawing on iStockPhoto. # 14th January 2010, 12:05 pm

GPS and Relativity (via) GPS satellite clock ticks need an accuracy of 20-30 nanoseconds. The satellites move fast enough that their clocks fall behind by 7 microseconds a day due to time dilation, but orbit high enough that the curvature of spacetime due to the Earth’s mass puts them forward by another 45 microseconds. GPS receivers have to perform relativistic calculations to determine their location! # 11th January 2010, 9:17 am

BLPOP and BRPOP in Redis. Added over Christmas—Redis now has blocking list pop operations. This means you can use Redis to drive a queue server without the need for polling—simply BLPOP against a key and, if it’s empty, your client will block until another client pushes an item on to the list. Multiple clients can block against the same key and only the first client will return when an item becomes available. # 7th January 2010, 10:50 pm

russell davies: datadecs. Personalised christmas decorations made from data from Twitter, Doppler, last.fm and Flickr. The Twitter snowman came from a 3D printer—the size of the head varies depending on your number of followers. Best of all though is the Flickr decoration which represents the apertures you’ve used over the past year. # 7th January 2010, 9:58 pm

What’s new in Django 1.2 alpha 1 (via) Multiple database support, improved CSRF prevention, a messages framework (similar to the Rails “flash” feature), model validation, custom e-mail backends, template caching for much faster handling of the include and extends tags, read only fields in the admin, a better if tag and more. Very exciting release. # 7th January 2010, 7:31 pm

last.fm for television. Dale Lane’s neat hack to visualise his television watching habits. An Ubuntu / vdx home theatre stores TV events in SQLite, and graphs are generated using Python and Open Flash Chart 2. The really clever bit: the back-end captures nearby bluetooth IDs’ allowing events to be filtered by the people watching based on the presence of their mobile phones. # 7th January 2010, 7:28 pm

Current State of Unladen Swallow (Towards a Faster Python). The Unladen Swallow team are now planning to merge their work back in to the mainline Python 3 branch, adding a powerful incentive for Python developers to port their old Python 2 code (since the official Python 2.x line is extremely unlikely to have Unladen Swallow merged in to it). # 7th January 2010, 12:41 pm

The Maximal Usage Doctrine for Open Source. Yehuda Katz shares my own philosophy on Open Source licensing—stick BSD or MIT on it to maximise the number of people who can use it. The projects I work on are small enough that I don’t care if someone makes big private improvements and refuses to share them. I can see how much larger projects like Linux would disagree though. # 6th January 2010, 5:23 pm