Simon Willison’s Weblog

Items tagged python in 2010

Filters: Year: 2010 × python ×


HotQueue. A super-simple Python work queue using Redis. The API is neat, and makes clever use of generators for blocking consumption of queue items. # 22nd December 2010, 11:51 am

What are the tradeoffs (e.g. development speed, performance, scalability) between using various php frameworks, ruby/rails, or python/django?  Is there any reason to choose one overwhelmingly over another?

At this point, I’d argue that the decision between them comes down to programming language rather than framework—the frameworks have mostly converged on a very similar set of features.

[... 145 words]

Who are the top Python programmers on Twitter?

Our site Lanyrd catalogs conference speakers by their Twitter ID, so you could use it to find people who speak at Python conferences—for example:

[... 52 words]

Is there a blog that covers open source Python projects?

No, but I wish there was.

[... 29 words]

Why does Python load imported modules separately for different files, unlike C or PHP? Isn’t that inefficient in terms of memory usage?

It doesn’t—you’re misunderstanding how Python’s module system works. If two different places have “import os” in them, the os module is only imported and executed once—it’s cached in the sys.modules dictionary so you can see it happen if you want to. The key thing to understand is that “import os” attaches the os module to the “os” symbol within the current file’s scope, loading it only if it hasn’t been loaded already.

[... 104 words]

What’s a good book about basic usage techniques and patterns in Python? (a la Effective Java/C++)

Dive into Python 3 is well worth a look: http://diveintopython3.org/

[... 48 words]

Bleach, HTML sanitizer and auto-linker. HTML sanitisation is notoriously difficult to do correctly, but Bleach (a Python library) looks like an excellent effort. It uses the html5lib parsing library to deal with potentially malformed HTML, uses a whitelist rather than a blacklist and has a neat feature for auto-linking URLs that is aware of the DOM (so it won’t try to auto-link a URL that is already wrapped in a link element). It was written by the Mozilla team for addons.mozilla.org and support.mozilla.org so it should be production ready. # 25th October 2010, 1:32 pm

What does the key word “yield” in Python do?

The best documentation I’ve seen on generators is this: http://www.dabeaz.com/generators/

[... 26 words]

Who are the best Python developers in Los Angeles?

Mahalo is based in Santa Monica, and they have a very talented team of Python/Django people. They also host a regular Django meetup: http://www.meetup.com/ladjango/

[... 40 words]

What are some tips for a new Python developer coming from C++/Java?

Read “Python is not Java” from 2004—it has some good tips on how working with the two languages differs.

[... 41 words]

Is the complexity of basic Python data structure operations documented?

Python’s C code has some excellent documentation, including detailed write-ups of how Python’s dictionary and sorting algorithms work. They’re a great read.

[... 42 words]

What’s the easiest way to take an existing Django view and make it “real time” via long polling?

I’d advocate decoupling your long polling endpoints entirely from the rest of your web app stack. Personally I like Node.js for this, but Tornado would work just fine too (I’ve experimented successfully with Tornado long polling in the past).

[... 171 words]

I think that “bad technology” can kill a startup, but slightly different variations of good technology don’t have much effect. Choose what you know/like best. And Ruby and Python are both in this latter category.

enko on Hacker News # 2nd October 2010, 11:19 am

ijson. A SAX-style streaming JSON parser for Python, using ctypes to talk to the yajl C library. # 22nd September 2010, 9:59 pm

Does SQLAlchemy depend on MySQLdb?

No. SQLAlchemy can talk to all sorts of different DB-API compliant backends, including MySQL Connector/J (Jython only), MySQL Connector/Python, mysql-python (the MySQLdb module) and OurSQL—plus backends for many other databases. See the full list here:

[... 50 words]

What is the best way to learn about setting up server software for Python based web apps?

I’m a big fan of Fabric for automated deployment scripts. Start by reading this tutorial: http://morethanseven.net/2009/07...

[... 40 words]

What is the highest traffic website built on top of Django?

My best guess would be Disqus. Instagram are pretty enormous these days as well.

[... 31 words]

Writing your own traceroute in Python. How to implement traceroute in Python, using the low-level socket module. # 9th August 2010, 12:58 pm

What to do when PyPI goes down. My deployment scripts tend to rely on PyPI these days (they install dependencies in to a virtualenv) which makes me distinctly uncomfortable. Jacob explains how to use the PyPI mirrors that are starting to come online, but that won’t help if the PyPI listing links to an externally hosted file which starts to 404, as happened with the python-openid package quite recently (now fixed). The comments on the post discuss workarounds, including hosting your own PyPI mirror or bundling tar.gz files of your dependencies with your project. # 21st July 2010, 10:19 am

simplegeo’s python-oauth2. The Python OAuth library scene is frighteningly complicated at the moment. This seems to be the most actively maintained, and the readme includes working example code for talking to the Twitter API (including integration with Django auth). # 18th July 2010, 5:22 pm

MapOSMatic. Clever service built on top of OpenStreetMap, which renders double sided city maps with a map and grid on one size and an A-Z street name index on the other. Runs on top of Mapnik, PostGIS and Cairo, with a few thousand additional lines of Python and Django. # 11th July 2010, 12:15 pm

python/trunk/Lib/httplib.py in 1994 (via) Python’s original httplib implementation, checked in by Guido 16 years and 4 months ago. Not much younger than the Web itself. # 4th July 2010, 11:25 pm

Python 2.7 Release. Includes three of my favourite features from Python 3: odicts, set literals and set and dictionary comprehensions. # 4th July 2010, 11:21 pm

Slide, Inc.—open source. slide.com have open sourced a whole bunch of interesting Python libraries, most of them involving C extensions or greenlet non-blocking I/O. wirebin (fast binary serialization of native Python types) and meminfo (an extension for finding precise in-memory sizes of Python objects) look particularly interesting. No documentation yet—not even a readme. # 17th June 2010, 8:05 pm

Appending the request URL to SQL statements in Django. A clever frame-walking monkey-patch which pulls the most recent HttpRequest object out of the Python stack and adds the current request.path to each SQL query as an SQL comment, so you can see it in debugging tools such as slow query logs and the PostgreSQL “select * from pg_stat_activity” query. # 2nd June 2010, 9:09 am

django-boss (via) Management commands are one of the few bits of Django that I still have to look up in the documentation whenever I write them. django-boss offers a smart alternative to regular management commands, based around decorators and taking the containing app as the first argument. # 1st June 2010, 10:02 am

Headroid1—a face tracking robot head. Kind of creepy—Ian Ozsvald’s openCV + pySerial motorised camera follows your face around the room, and will soon be able to react to your emotions. # 21st May 2010, 4:59 pm

Django 1.2 release notes (via) Released today, this is a terrific upgrade. Multiple database connections, model validation, improved CSRF protection, a messages framework, the new smart if template tag and lots, lots more. I’ve been using the 1.2 betas for a major new project over the past few months and it’s been smooth sailing all the way. # 17th May 2010, 9:11 pm

Ruby-style Blocks in Python. Yes, yes, yes, yes. A proposal for muli-line lambda support in Python that doesn’t trip up on significant whitespace. If this gets in before the proposed feature freeze I’ll be a very happy Pythonista. UPDATE: This is a post from over a year ago, and it looks like the proposal has since stalled. # 23rd April 2010, 11:19 am

Flask 0.1 Released. Armin’s Flask (a Python microframework built around Werkzeug and Jinja2) is looking pretty solid for a two week old project—extensive documentation, comprehensive unit test support (and example applications with unit tests) and some very tidy API design. # 16th April 2010, 5:12 pm