Simon Willison’s Weblog

What is the highest traffic website built on top of Django?

My best guess would be Disqus. Instagram are pretty enormous these days as well.

What is the history of the Django web framework? Why has it been described as “developed in a newsroom”?

I was there!

Getting married and going travelling

It’s been a busy month. On Saturday the 5th of June I married the wonderful Natalie Downe in a beautiful ceremony at Roedean School in Brighton. The reception had owls, cheese, a ferret, a golden eagle, amazing Turkish food, Jewish chair dancing and lovely guests. It was the happiest day of my life.

Comprehensive notes from my three hour Redis tutorial

Last week I presented two talks at the inaugural NoSQL Europe conference in London. The first was presented with Matthew Wall and covered the ways in which we have been exploring NoSQL at the Guardian. The second was a three hour workshop on Redis, my favourite piece of software to have the NoSQL label applied to it.

WildlifeNearYou talk at £5 app, and being Wired (not Tired)

Two quick updates about WildlifeNearYou. First up, I gave a talk about the site at £5 app, my favourite Brighton evening event which celebrates side projects and the joy of Making Stuff. I talked about the site’s genesis on a fort, crowdsourcing photo ratings, how we use Freebase and DBpedia and how integrating with Flickr’s machine tags gave us a powerful location API for free. Here’s the video of the talk, courtesy of Ian Oszvald:

Some questions about the “blocking” of HTML5

Some background reading. I was planning to fill in answers as they arrive, but I screwed up the moderation of the comments and got flooded with detailed responses—I strongly recommend reading the comments.

WildlifeNearYou: It began on a fort...

Back in October 2008, myself and 11 others set out on the first /dev/fort expedition. The idea was simple: gather a dozen geeks, rent a fort, take food and laptops and see what we could build in a week.

Crowdsourced document analysis and MP expenses

As you may have heard, the UK government released a fresh batch of MP expenses documents a week ago on Thursday. I spent that week working with a small team at Guardian HQ to prepare for the release. Here’s what we built:

Node.js is genuinely exciting

I gave a talk on Friday at Full Frontal, a new one day JavaScript conference in my home town of Brighton. I ended up throwing away my intended topic (JSONP, APIs and cross-domain security) three days before the event in favour of a technology which first crossed my radar less than two weeks ago.

Why I like Redis

I’ve been getting a lot of useful work done with Redis recently.

This shouldn’t be the image of Hack Day

I love hack days. I was working in the vicinity of Chad Dickerson when he organised the first internal Yahoo! Hack Day back in 2005, and I’ve since participated in hack day events at Yahoo!, Global Radio and the Guardian. I’ve also been to every one of Yahoo!’s Open Hack Day events in London. They’re fantastic, and the team that organises them should be applauded.

Django ponies: Proposals for Django 1.2

I’ve decided to step up my involvement in Django development in the run-up to Django 1.2, so I’m currently going through several years worth of accumulated pony requests figuring out which ones are worth advocating for. I’m also ensuring I have the code to back them up—my innocent AutoEscaping proposal a few years ago resulted in an enormous amount of work by Malcolm and I don’t think he’d appreciate a repeat performance.

Hack Day tools for non-developers

We’re about to run our second internal hack day at the Guardian. The first was an enormous amount of fun and the second one looks set to be even more productive.

Teaching users to be secure is a shared responsibility

Ryan Janssen: Why an OAuth iframe is a Great Idea.

Facebook Usernames and OpenID

Today’s launch of Facebook Usernames provides an obvious and exciting opportunity for Facebook to become an OpenID provider. Facebook have clearly demonstrated their interest in becoming the key online identity for their users, and the new usernames feature is their acknowledgement that URL-based identities are an important component of that, no doubt driven in part by Twitter making usernames trendy again.

djng—a Django powered microframework

djng is nearly two weeks old now, so it’s about time I wrote a bit about the project.

rev=canonical bookmarklet and designing shorter URLs

I’ve watched the proliferation of URL shortening services over the past year with a certain amount of dismay. I care about the health of the web and try to ensure that URLs I am responsible will last for as long as possible, and I think it’s very unlikely that all of these new services will still be around in twenty years time. Last month I suggested that the Internet Archive start mirroring redirect databases, and last week I was pleased to hear that Archiveteam, a different organisation, had already started crawling.

List of SxSW 2009 panels with “social” in the title

• A Hard Sell? Social Media & Your Boss
• Can Social Media End Racism?
• Digital Urbanites: How To Become Part of the New Social Capital
• The Future Of Social Networks
• How Social Networks Are Killing the Revolution
• Making Whuffie: Raising Social Capital in Online Communities
• The Mix at Six Hosted by Social Media Group
• Mobile Social SXSW BBQ
• My Boss Doesn’t Get It: Championing Social Media to the Man
• PBS’ Interactive Social Media & Online Video Studio
• The Search for a More Social Web
• Security for the Social Set
• Social Engineering: Scam Your Way Into Anything or From Anybody
• Social Gamers: Away From the Keyboard
• Social Media For Social Good
• Social Media Marketing
• Social Media Marketing: An Hour a Day
• Social Media Nonprofit ROI Poetry Slam
• Social Media: If You Liked it, Then You Should Have Put a Digg on it...
• Social Networking in Health: e-Patients, Data & Privacy
• Social Patterns and Antipatterns For the Win
• Suxorz ’09: The Ten Worst Social Media Campaigns
• Twitter for Marketers: Is It Still Social Media?
• Using GPS & Location to Enhance Social Networking
• Using the New Digital Social Media to Accelerate Sustainability

A few notes on the Guardian Open Platform

This morning we launched the Guardian Open Platform at a well attended event in our new offices in Kings Place. This is one of the main projects I’ve been helping out with since joining the Guardian last year, and it’s fantastic to finally have it out in the open.

Pragmatism, purity and JSON content types

I started a conversation about this on Twitter the other day, but Twitter is a horrible place to have an archived discussion so I’m going to try again here.

Rate limiting with memcached

On Monday, several high profile “celebrity” Twitter accounts started spouting nonsense, the victims of stolen passwords. Wired has the full story—someone ran a dictionary attack against a Twitter staff member, discovered their password and used Twitter’s admin tools to reset the passwords on the accounts they wanted to steal.

DjangoCon and PyCon UK

September is a big month for conferences. DjangoCon was a weekend ago in Mountain View (forcing me to miss both d.Construct and BarCamp Brighton), PyCon UK was this weekend in Birmingham, I’m writing this from @media Ajax and BarCamp London 5 is coming up over another weekend at the end of this month. As always, I’ve been posting details of upcoming talks and notes and materials from previous ones on my talks page.

Announcing dmigrations

The team at Global Radio (formerly GCap Media) is the largest group of Django developers I’ve personally worked with, consisting of 14 developers split into two scrum teams, all contributing to the same overall codebase.

Back to full-time employment

I’ve been freelance for a year and a half now, and it’s been a great deal of fun. For me, being freelance meant having the freedom to pursue all sorts of different interests—technical writing, public speaking, Django, OpenID, JavaScript—and the opportunity to work with some really fantastic people.

The point of “Open” in OpenID

TechCrunch report that Microsoft are accepting OpenID for their new HealthVault site, but with a catch: you can only use OpenIDs from two providers: Trustbearer (who offer two-factor authentication using a hardware token) and Verisign. "Whatever happened to the Open in OpenID?", asks TechCrunch’s Jason Kincaid.

Debugging Django

I gave a talk on Debugging Django applications at Monday’s inaugural meeting of DJUGL, the London Django Users Group. I wanted to talk about something that wasn’t particularly well documented elsewhere, so I pitched the talk as “Bug Driven Development”—what happens when Test Driven Development goes the way of this unfortunate pony.

jQuery style chaining with the Django ORM

Django’s ORM is, in my opinion, the unsung gem of the framework. For the subset of SQL that’s used in most web applications it’s very hard to beat. It’s a beautiful piece of API design, and I tip my hat to the people who designed and built it.

wikinear.com, OAuth and Fire Eagle

I’m pleased to announce wikinear.com. It’s a simple site that does just one thing: show you a list of the five Wikipedia pages that are geographically closest to your current location. It’s designed (or not-designed) to be used mainly from mobile phones.

Django People: OpenID and microformats

In hindsight, it was a mistake to launch Django People without support for OpenID. It was on the original feature list, but in the end I decided to cut any feature that wasn’t completely essential in order to get the site launched before it drowned in an ocean of “wouldn’t-it-be-cool-ifs”.

Django People

I’m constantly surprised by the number of people I run in to at conferences (or even in one case on the train) who are using Django but are completely invisible to the Django community. It seems that this is the downside of having good documentation: many people just read it and start building, without ever showing their face on the mailing lists or IRC.