Simon Willison’s Weblog

Blogmarks tagged django, python

Filters: Type: blogmark × django × python ×


Pysa: An open source static analysis tool to detect and prevent security issues in Python code (via) Interesting new static analysis tool for auditing Python for security vulnerabilities—things like SQL injection and os.execute() calls. Built by Facebook and tested extensively on Instagram, a multi-million line Django application. # 7th August 2020, 8:50 pm

PEP 8016 -- The Steering Council Model (via) The votes are in and Python has a new governance model, partly inspired by the model used by the Django Software Foundation. A core elected council of five people (with a maximum of two employees from any individual company) will oversee the project. # 17th December 2018, 4:02 pm

Channels 2.0. Andrew just shipped Channels 2.0—a major rewrite and redesign of the Channels project he started back in 2014. Channels brings async to Django, providing a logical, standardized way of supporting things like WebSockets and asynchronous execution on top of a Django application. Previously it required you to run a separate Twisted server and redis/RabbitMQ queue, but thanks to Python 3 async everything can now be deployed as a single process. And the new ASGI spec means its turtles all the way down! Everything from URL routing to view functions to middleware can be composed together using the same ASGI interface. # 2nd February 2018, 6:19 pm

Using setup.py in Your (Django) Project. Includes this neat trick: if you list manage.py in the setup(scripts=) argument you can call it from e.g. cron using the full path to manage.py within your virtual environment and it will execute in the correct context without needing to explicitly activate the environment first. # 2nd February 2018, 12:33 pm

Building a statistical profiler in python. Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful. # 5th October 2017, 3:44 pm

The First Few Weeks—ep.io. Another take on managed Python Django/WSGI hosting, from Andrew Godwin and Ben Firshman. # 13th January 2011, 4:25 am

Bleach, HTML sanitizer and auto-linker. HTML sanitisation is notoriously difficult to do correctly, but Bleach (a Python library) looks like an excellent effort. It uses the html5lib parsing library to deal with potentially malformed HTML, uses a whitelist rather than a blacklist and has a neat feature for auto-linking URLs that is aware of the DOM (so it won’t try to auto-link a URL that is already wrapped in a link element). It was written by the Mozilla team for addons.mozilla.org and support.mozilla.org so it should be production ready. # 25th October 2010, 1:32 pm

simplegeo’s python-oauth2. The Python OAuth library scene is frighteningly complicated at the moment. This seems to be the most actively maintained, and the readme includes working example code for talking to the Twitter API (including integration with Django auth). # 18th July 2010, 5:22 pm

MapOSMatic. Clever service built on top of OpenStreetMap, which renders double sided city maps with a map and grid on one size and an A-Z street name index on the other. Runs on top of Mapnik, PostGIS and Cairo, with a few thousand additional lines of Python and Django. # 11th July 2010, 12:15 pm

Appending the request URL to SQL statements in Django. A clever frame-walking monkey-patch which pulls the most recent HttpRequest object out of the Python stack and adds the current request.path to each SQL query as an SQL comment, so you can see it in debugging tools such as slow query logs and the PostgreSQL “select * from pg_stat_activity” query. # 2nd June 2010, 9:09 am

django-boss (via) Management commands are one of the few bits of Django that I still have to look up in the documentation whenever I write them. django-boss offers a smart alternative to regular management commands, based around decorators and taking the containing app as the first argument. # 1st June 2010, 10:02 am

Django 1.2 release notes (via) Released today, this is a terrific upgrade. Multiple database connections, model validation, improved CSRF protection, a messages framework, the new smart if template tag and lots, lots more. I’ve been using the 1.2 betas for a major new project over the past few months and it’s been smooth sailing all the way. # 17th May 2010, 9:11 pm

Introduction to Surlex. A neat drop-in alternative for Django’s regular expression based URL parsing, providing simpler syntax for common path patterns. # 11th April 2010, 7:23 pm

The Onion Uses Django, And Why It Matters To Us. The Onion ported their main site from PHP and Drupal to Django in three months with a team of four developers, including a full migration of their archived content. Their developers answer questions about the switch in this thread on the Django sub-reddit. # 25th March 2010, 6:43 pm

Cache Machine: Automatic caching for your Django models. This is the third new ORM caching layer for Django I’ve seen in the past month! Cache Machine was developed for zamboni, the port of addons.mozilla.org to Django. Caching is enabled using a model mixin class (to hook up some post_delete hooks) and a custom caching manager. Invalidation works by maintaining a “flush list” of dependent cache entries for each object—this is currently stored in memcached and hence has potential race conditions, but a comment in the source code suggests that this could be solved by moving to redis. # 11th March 2010, 7:35 pm

Is johnny-cache for you? “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. # 2nd March 2010, 11:44 am

jmoiron.net: Johnny Cache. The blog entry announcing Johnny Cache (“a drop-in caching library/framework for Django that will cache all of your querysets forever in a consistent and safe manner”) to the world. # 1st March 2010, 11:48 am

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. # 28th February 2010, 10:55 pm

jacobian’s django-deployment-workshop. Notes and resources from Jacob’s 3 hour Django deployment workshop at PyCon, including example configuration files for Apache2 + mod_wsgi, nginx, PostgreSQL and pgpool. # 19th February 2010, 2:28 pm

Why toppcloud will not be agnostic. Ian Bicking’s toppcloud aims to offer deployment with the ease of use of AppEngine against a standard, open source Ubuntu + Python 2.6 + mod_wsgi + Varnish stack. Here he explains why he’s not going to vary the required components: keeping everything completely standardised means everyone gets the same bugs (and the same fixes). # 12th February 2010, 9:21 am

Presenting django-devserver, a better runserver. I really like this—it’s a Django management command (./manage.py rundevserver) which adds SQL logging and cache access summaries to the console output of Django’s local development server. It solves a similar set of problems to the debug toolbar, but requires slightly less setup and doesn’t inject additional HTML in to your pages. You can add your own custom modules to it as well. # 10th February 2010, 11:33 am

Integrate Tornado in Django. A handy ./manage.py runtornado management command for firing up a Tornado server that serves your Django application. # 8th February 2010, 11:12 am

World Government Data. Launched last week, this is the Guardian’s meta-search engine for searching and browsing through data from four different government data sites (with more sites planned). Under the hood it’s Django, Solr, Haystack and the Scrapy crawling library. The application was built by Ben Firshman during an internship over Christmas. # 27th January 2010, 12:27 pm

For the Love of Ponies. Bryan Veloso, the discoverer of the Django Pony, makes contact with the artist who released the original drawing on iStockPhoto. # 14th January 2010, 12:05 pm

What’s new in Django 1.2 alpha 1 (via) Multiple database support, improved CSRF prevention, a messages framework (similar to the Rails “flash” feature), model validation, custom e-mail backends, template caching for much faster handling of the include and extends tags, read only fields in the admin, a better if tag and more. Very exciting release. # 7th January 2010, 7:31 pm

Timing attack in Google Keyczar library. An issue I also need to fix in the proposed Django signing code. If you’re comparing two strings in crypto (e.g. seeing if the provided signature matches the expected signature) you need to use a timing independent string comparison function or you risk leaking information. This kind of thing is exactly why I want an audited signing module in Django rather than leaving developers to figure it out on their own. # 4th January 2010, 3:23 pm

Design and code review requested for Django string signing / signed cookies. Do you know your way around web app security and cryptography (in particular signing things using hmac and sha1)? We’d appreciate your help reviewing the usage of these concepts in Django’s proposed string signing and signed cookie implementations. # 4th January 2010, 1:24 pm

Videos from DjangoCon 2009. The videos from September’s DjangoCon are now available, including my “Cowboy development with Django” talk. # 3rd January 2010, 11:02 am

Django | Multiple Databases. Russell just checked in the final patch developed from Alex Gaynor’s Summer of Code project to add multiple database support to Django. I’d link to the 21,000 line changeset but it crashed our Trac, so here’s the documentation instead. # 22nd December 2009, 5:22 pm

Django-Jython 1.1.0 released. Django on Jython matches its minor version numbers to Django, so this new release is compatible with Django 1.1. # 16th December 2009, 10:42 pm