Simon Willison’s Weblog

Items tagged python in 2009

Filters: Year: 2009 × python ×

Web Sockets in Tornado. Bret Taylor has a simple class making it trivial to experiment with the Web Sockets protocol (now shipping in Chrome) using the scalable Tornado application server. He also raises the million dollar question: what will existing load balancers and proxies make of the new protocol? # 31st December 2009, 11:54 am

Socket Benchmark of Asynchronous Servers in Python. A comparison of eight different asynchronous networking frameworks in Python. Tornado comes out on top in most of the benchmarks, but the post is most interesting for the direct comparison of simple code examples for each of the frameworks. # 22nd December 2009, 10:34 pm

Django | Multiple Databases. Russell just checked in the final patch developed from Alex Gaynor’s Summer of Code project to add multiple database support to Django. I’d link to the 21,000 line changeset but it crashed our Trac, so here’s the documentation instead. # 22nd December 2009, 5:22 pm

Crowdsourced document analysis and MP expenses

As you may have heard, the UK government released a fresh batch of MP expenses documents a week ago on Thursday. I spent that week working with a small team at Guardian HQ to prepare for the release. Here’s what we built:

[... 2051 words]

Django-Jython 1.1.0 released. Django on Jython matches its minor version numbers to Django, so this new release is compatible with Django 1.1. # 16th December 2009, 10:42 pm

Guardian iPhone app. Released today, ad-free, £2.39 for the application, has an excellent offline mode. I helped build the backend web service, which is a Django app running on EC2. # 14th December 2009, 1:29 pm

Fixing Django Management Commands. Zachary Voase proposes dramatically improving Django’s management command API for Django 1.3. I’m in favour—management commands are one of the only APIs in Django that I have to look up every single time I use. My optfunc library was written partially with management commands in mind—Zachary favours the argparse library. # 9th December 2009, 8:41 am

Python’s Moratorium—Let’s think about this. Jesse Noller explains the thinking behind the Python Language Moratorium (no new language features until Python 3.3) in great detail. It’s principally about allowing both end users and alternative implementations to catch up. The standard library will continue to evolve as normal. # 5th December 2009, 5:33 pm

What’s coming in Django 1.2 (presentation notes). I wrote up some background notes for the talk on Django 1.2 I gave at DJUGL last week. # 5th December 2009, 5:10 pm

Namespaces. Python’s approach to imports is possibly my favourite feature of the language. I love being able to scan up to the top of a file in my text editor and see exactly where every symbol comes from, no IDE required. # 2nd December 2009, 9:31 am

Haystack 1.0 Final Released. I’ve used Haystack on a number of projects recently, and it has proved itself as a completely painless way of adding full-text search (using Solr or Whoosh—I haven’t tried the Xapian backend yet) to a Django ORM powered project in just a few minutes. Congratulations, Daniel + contributors. # 30th November 2009, 8:07 am

Djangopeople JSON parser. Awesome—Andy McKay has compensated for the lack of an official DjangoPeople API by creating a JSONP screen scraped API and hosting it on App Engine. As far as I’m concerned this is an officially supported feature—I’ll make sure future site changes don’t break it, and when I do add an API I’ll try to keep it compatible and help Andy set up redirects. # 28th November 2009, 11:29 am

Request Routing With URI Templates in Node.JS. I quite like this approach (though the implementation is a bit “this” heavy for my taste). JavaScript has no equivalent to Python’s raw strings, so regular expression based routing ala Django ends up being a bit uglier in JavaScript. URI template syntax is more appealing. # 24th November 2009, 9:06 am

Woof—simply exchange files (via) Ultra simple file sharing for local networks: run “woof filename” to start a local web server which will serve up that file, just once, and then terminate. Can also serve up an entire directory as a compressed archive. Written in Python, as a single script which you can drop in to your ~/bin. “woof -s” serves the script itself, so you can easily pass it to someone who has a file you want. # 24th November 2009, 8:44 am

django-batch-select (via) A smart attempt at solving select_related for many-to-many relationships in Django. Add a custom manager to your model and call e.g. Entry.objects.all()[:10].batch_select(“tags”) to execute two queries—one pulling back the first ten entries and another using an “IN” query against the tags table to pull back all of the tags for those entries in one go. # 23rd November 2009, 4:19 pm

Debugging in Python. The missing manual for Python’s powerful pdb debugger. # 18th November 2009, 12:34 pm

Drupal or Django? A Guide for Decision Makers. A surprisingly interesting comparison—the author describes Django as “a framework with CMS-like tendencies” and Drupal as “a CMS with framework-like tendencies”, then explores the benefits of those two different approaches. # 15th November 2009, 10:14 pm

How to Make a US County Thematic Map Using Free Tools. This is the trick I’ve been using to generate choropleths at the Guardian for the past year: figure out the preferred colours for a set of data in a Python script and then rewrite an SVG file to colour in the areas. I use ElementTree rather than BeautifulSoup but the technique is exactly the same. The best thing about SVG is that our graphics department can export them directly out of Illustrator, with named layers and paths automatically becoming SVG ID attributes. Bonus tip: sometimes you don’t have to rewrite the SVG XML at all, instead you can generate CSS to colour areas by ID selector and inject it in to the top of the file. # 12th November 2009, 10:49 am

Writing good documentation (part 1). Jacob explains some of the philosophy behind Django’s documentation. Topical guides are particularly interesting—many projects skip them (leaving books to fill the gap) but they fill an essential gap between tutorials and low-level reference documentation. # 11th November 2009, 7:13 am

A history of Python packaging. A comprehensive history by Martijn Faassen, who argues that the existing set of tools tools works fine and has been working fine for several years. # 10th November 2009, 8:48 pm

Fabric 0.9.0. A Python-based SSH automation and deployment tool. Released today, 0.9.0 is finally the official “stable” release—which is good, as it breaks API compatibility with previous versions and caused me all sorts of confusion when I tried to learn Fabric recently. # 9th November 2009, 2:02 pm

Django-Jython 1.0.0 released! Now with database backends for PostgreSQL, Oracle and MySQL. The next release (planned for next month) should provide full compatibility with Django 1.1—the current release has 1.1 support for PostgreSQL but only 1.0 support for the other two databases. # 9th November 2009, 1:53 pm

Python in the Scientific World. Python continues to make strides in the scientific world—and the Hubble Space Telescope team have been using it for 10 years! # 6th November 2009, 11:04 am

Frank Wierzbicki: Leaving Sun. Frank performed miracles at Sun and before, helping bring the Jython project out of stasis and turning it in to an active, community maintained modern Python implementation. If you’re looking for an expert Python/Java/Dynamic languages guy you should snap him up. # 4th November 2009, 10:33 pm

Large Problems in Django, Mostly Solved: Search. Eric Holscher shows how Haystack uses a number of common Django patterns (object registration, pluggable backends, QuerySet-style chaining and class-based views) to great effect in creating a powerful search application for Django. Makes me wonder if more of those patterns should be promoted to first class concepts within Django. # 3rd November 2009, 10:42 am

Exploring Python (via) Notes from the introduction to Python presentation I gave today at Stack Overflow DevDays Amsterdam. # 2nd November 2009, 3:35 pm

pudb. A full-screen, curses console based visual debugger for Python, built using the urwid console UI library. # 1st November 2009, 12:09 pm

Django 1.2 planned features. The votes are in and the plan for Django 1.2 has taken shape—features are split in to high, medium and low priority. There’s some really exciting stuff in there—outside of the things I’ve already talked about, I’m particularly excited about multidb, Model.objects.raw(SQL), the smarter {% if %} tag and class-based generic views. # 26th October 2009, 10:38 am

Twisted inlineCallbacks and deferredGenerator. inlineCallbacks are a brilliant (but seemingly under-promoted) feature of Twisted which use the ability to return a value from a yield statement to make asynchronous callbacks look much more like regular sequential programming. # 25th October 2009, 11:30 pm

Why I like Redis

I’ve been getting a lot of useful work done with Redis recently.

[... 900 words]