OpenIDRecent entries
DjangoCon and PyCon UK three months ago
September is a big month for conferences. DjangoCon was a weekend ago in Mountain View (forcing me to miss both d.Construct and BarCamp Brighton), PyCon UK was this weekend in Birmingham, I’m writing this from @media Ajax and BarCamp London 5 is coming up over another weekend at the end of this month. As always, I’ve been posting details of upcoming talks and notes and materials from previous ones on my talks page.
DjangoCon went really, really well. Huge thanks to conference chair Robert Lofthouse for pulling it all together in just two months and Leslie Hawthorne for making it all happen from Google’s end. Google’s facitilies were superb: the AV team were the best I’ve ever worked with and an army of Google volunteers made sure everything went smoothly. It’s hard to see how it could have gone better; the principle complaint we got was that at only two days it was hard to justify the travel, something which future DjangoCons will definitely address.
Every session was recorded and the videos should be going up on YouTube shortly are now up on YouTube. For the impatient, you can subscribe to an Atom feed of a YouTube search for “DjangoCon”. I recommend starting with Cal Henderson’s keynote “Why I hate Django” which was both funny and insightful in equal parts. Malcolm’s talk on ORM internals was another personal favourite.
PyCon UK was the second I’ve attended, but last year I only stayed for the first day. This time I stuck around and was enormously impressed by the grassroots feel of the conference and the enthusiastic atmosphere. I presented a tutorial on extending the Django admin and a lightning talk on Zeppelins, prepared two hours in advance after Jacob mentioned that the lightning talks were tending too much towards the technical side. It went down very well; I’m tempted to extend it to a half hour session for BarCamp London.
Unlike most conferences I attend, PyCon tickets included a sit-down dinner for all attendees complete with a “dramatic lecture” on the Lunar Society presented by Andrew Lound. This was a great fit for the conference, both for the Birmingham connection and the many analogies to the modern open source community—loose collaboration, patent concerns and what you might call an 18th century equivalent of the modern hacker ethic.
Next year the PyCon UK team will be hosting EuroPython, and I’m certain they’ll do an excellent job of it. Meanwhile, Rob has already started making plans for a Euro DjangoCon in around six months time, probably taking place in Prague.
Announcing dmigrations four months ago
The team at Global Radio (formerly GCap Media) is the largest group of Django developers I’ve personally worked with, consisting of 14 developers split into two scrum teams, all contributing to the same overall codebase.
Working with that many developers makes smart tools and processes essential, and in some cases we’ve had to develop our own. Today, we’re releasing one of them as an open source project.
dmigrations is a Django migrations tool. It addresses a common problem in Django development: if you change a model after creating the database tables for it with syncdb, how do you reflect those changes in your database tables without blowing away your existing data and starting again from scratch?
django-evolution attempts to address this problem the clever way, by detecting changes to models that are not yet reflected in the database schema and figuring out what needs to be done to bring the two back in sync. In contrast, dmigrations takes the stupid approach: it requires you to explicitly state the changes in a sequence of migrations, which will be applied in turn to bring a database up to the most recent state that reflects the underlying models.
This means extra work for developers who create migrations, but it also makes the whole process completely transparent—for our projects, we decided to go with the simplest system that could possibly work.
The interface to dmigrations is a pair of custom Django commands. The first, ./manage.py dmigrate, provides a set of command for listing, applying and unapplying (reverting) migrations. This entirely replaces Django’s syncdb command.
The second, ./manage.py dmigration, provides commands for code-generating new migrations. It turns out that most migrations fit a set of common patterns: add a new table, add the tables for a new Django application, add a column to an existing table, add an index. These common cases are handled by dmigration; if you want to do something more complex (rename a column while transforming its data for example) you’ll need to write a custom migration class.
The dmigrations tutorial provides a full introduction to both of these commands, as well as hints on writing your own custom migrations. Since migrations are just classes, one of our hopes is that external developers will write extra migration classes for operations like “rename column”—things that currently require a one-off custom migration.
dmigrations is actually the third iteration of our in-house migrations system. The first, smigrations, was designed to do the least amount of work possible to give us a controlled way of applying changes to our database schemas. The ’s’ stood for ’simple’. The second version (dmigrations) written by Tomasz Wegrzanowski consisted of a major upgrade to smigrations that addressed many of the frustrations we found when using it with branched development, in particular the problem of migrations in two branches conflicting with each other. The ’d’ stood for ’distributed’.
Version three, released today, is my refactoring of dmigrations to de-couple it from the rest of our codebase. I’ve also stubbed out hooks for adding support for alternative database engines; dmigrations only supports MySQL out of the box, but I’m keen on getting it working with other databases now that it’s out in the wild. Patches welcome!
How does this fit in with South and django-evolution?
That’s an excellent question. We’ll be discussing all three systems on the schema evolution panel at DjangoCon this weekend. I would love to see co-operation between the projects; at the very least I’d like to see the emergence of a standard Django-style abstraction library for create/alter table statements (something we punted on entirely with dmigrations). You’ll certainly be hearing a lot more about migrations in Django after the conference.
Back to full-time employment four months ago
I’ve been freelance for a year and a half now, and it’s been a great deal of fun. For me, being freelance meant having the freedom to pursue all sorts of different interests—technical writing, public speaking, Django, OpenID, JavaScript—and the opportunity to work with some really fantastic people.
It was going to take a very special opportunity to pull me back in to full-time employment, but I believe I’ve found that opportunity at the Guardian. I’ll be joining them full time (well, four days a week) in mid-October as a software architect, collaborating with their development team on some ambitious API projects. The Guardian have access to a lot of interesting data and I can’t wait to get stuck in to it. Since they’re a newspaper, it shouldn’t be a surprise that they scooped me to the story.
I’ll be particularly sorry to say good-bye to the outstanding team I’ve been working with at GCap. I’m looking forward to talking about some of the things we’ve been working together over the next few weeks.
