Simon Willison’s Weblog

Items tagged django in Oct

Filters: Month: Oct × django ×

How I moderated the State of Django panel at DjangoCon US.

On Wednesday last week I moderated the State of Django panel as the closing session for DjangoCon US 2018.

[... 1210 words]

Squeezing every drop of performance out of a Django app on Heroku. Ben Firshman describes some lesser known tricks for scaling Django on Heroku—in particular, using gunicorn gevent asynchronous workers and setting up PostgreSQL connection pooling using django-db-geventpool. # 31st October 2017, 2:08 pm

arxiv-vanity (via) Beautiful new project from Ben Firshman and Andreas Jansson: “Arxiv Vanity renders academic papers from Arxiv as responsive web pages so you don’t have to squint at a PDF”. It works by pulling the raw LaTeX source code from Arxiv and rendering it to HTML using a heavily customized Pandoc workflow. The real fun is in the architecture: it’s a Django app running on Heroku which fires up on-demand Docker containers for each individual rendering job. # 25th October 2017, 8:06 pm

A Brief Intro to Docker for Djangonauts (via) This is great—a really clear introduction to both Docker and Docker Compose, aimed at Django developers. Includes line-by-line annotations of an example Dockerfile and docker-compose.yml. # 18th October 2017, 9:06 pm

How to set up world-class continuous deployment using free hosted tools

I’m going to describe a way to put together a world-class continuous deployment infrastructure for your side-project without spending any money.

[... 1294 words]

My essential django package list. Insightful list of Django packages—many of which I hadn’t seen before—by Serafeim Papastefanos, each with a handy explanation of what it’s useful for and why. # 11th October 2017, 2:42 pm

Generate dynamic titles for /search/ pages. Fun little enhancement to my faceted search implementation: I now generate dynamic titles for each search results page describing the search, e.g. “Blogmarks tagged security in Feb, 2005” or ““python” in quotations tagged ruby, python in 2007”. # 8th October 2017, 7:47 pm

Building a statistical profiler in python. Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful. # 5th October 2017, 3:44 pm

Implementing faceted search with Django and PostgreSQL

I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter by content type (entries vs blogmarks vs quotation) and any combination of the above. Some example searches:

[... 3049 words]

Running gunicorn behind nginx on Heroku for buffering and logging

Heroku’s default setup for Django uses the gunicorn application server. Each Heroku dyno can only run a limited number of gunicorn workers, which means a limited number of requests can be served in parallel (around 4 per dyno is a good rule of thumb).

[... 400 words]

For a Django application, deployed on Heroku, what are my options for storing user-uploaded media files?

S3 is really a no-brainer for this, it’s extremely inexpensive, very easy to integrate with and unbelievably reliable. It’s so cheap that it will be practically free for testing purposes (expect to spend pennies a month on it).

[... 88 words]

How can Google’s Zopfli compression algorithm be implemented in Django’s cache system?

It’s pretty easy to write custom backends for Django’s caching layer—check out for example.

[... 36 words]

Django (web framework): What is the recommended way to cache objects that are very large (>1.5MB)?

Take a look at redis—it can handle binary strings up to 512 MB and has performance that is similar to memcache. It’s very easy to use from within Django (there’s even a pluggable cache backend for it).

[... 61 words]

How can I learn more about server-side technologies?

Get yourself a VPS, set it up from scratch and run some non-critical websites on it (nothing with private user data since you can’t be sure you’ll set it up securely). Both Slicehost and Linode offer a good set of guides to a whole host of common tasks:

[... 109 words]

What is the best way to integrate MongoDB with Django?

Personally, I just “import pymongo” and start calling the regular Python API—no need for any special treatment to get it working with Django.

[... 41 words]

Bleach, HTML sanitizer and auto-linker. HTML sanitisation is notoriously difficult to do correctly, but Bleach (a Python library) looks like an excellent effort. It uses the html5lib parsing library to deal with potentially malformed HTML, uses a whitelist rather than a blacklist and has a neat feature for auto-linking URLs that is aware of the DOM (so it won’t try to auto-link a URL that is already wrapped in a link element). It was written by the Mozilla team for and so it should be production ready. # 25th October 2010, 1:32 pm

What does an ideal Django workflow setup look like?

Short answer: virtualenv, pip, south for migrations, fabric for deployment.

[... 57 words]

Who are the best Python developers in Los Angeles?

Mahalo is based in Santa Monica, and they have a very talented team of Python/Django people. They also host a regular Django meetup:

[... 40 words]

What’s the easiest way to take an existing Django view and make it “real time” via long polling?

I’d advocate decoupling your long polling endpoints entirely from the rest of your web app stack. Personally I like Node.js for this, but Tornado would work just fine too (I’ve experimented successfully with Tornado long polling in the past).

[... 171 words]

Django 1.2 planned features. The votes are in and the plan for Django 1.2 has taken shape—features are split in to high, medium and low priority. There’s some really exciting stuff in there—outside of the things I’ve already talked about, I’m particularly excited about multidb, Model.objects.raw(SQL), the smarter {% if %} tag and class-based generic views. # 26th October 2009, 10:38 am

Play framework for Java. I’m genuinely impressed by this—it’s a full stack web framework for Java that actually does feel a lot like Django or Rails. Best feature: code changes are automatically detected and reloaded by the development web server, giving you the same save-and-refresh workflow you get in Django (no need to compile and redeploy to try out your latest changes). # 25th October 2009, 11:21 pm

Introducing Cloudera Desktop. It’s a GUI for Hadoop, and under the hood is a whole stack of open source software, including Python, Django, MooTools, Twisted, lxml, CherryPy, Mako, Java and AspectJ. # 21st October 2009, 6:48 pm

nginx_http_push_module. More clever design with webhooks—here’s an nginx module that provides a comet endpoint URL which will hang until a back end process POSTs to another URL on the same server. This makes it much easier to build asynchronous comet apps using regular synchronous web frameworks such as Django, PHP and Rails. # 17th October 2009, 4:48 pm

Django security updates released. A potential denial of service vulnerability has been discovered in the regular expressions used by Django form library’s EmailField and URLField—a malicious input could trigger a pathological performance. Patches (and patched releases) for Django 1.1 and Django 1.0 have been published. # 10th October 2009, 12:24 am in Django. A seriously impressive case study—a complete rebuild from the ground up completed in just five weeks using Django, Solr and Haystack for a high traffic site with a top 10,000 US Alexa ranking. # 9th October 2009, 12:38 am

History of Django’s popularity. “What sequence of events made Django the most popular Python web framework?”—insightful answers from Alex Martelli and James Bennett. # 4th October 2009, 10:29 am

MySQL Connector/Python. A pure Python implementation of the MySQL client/server protocol, meaning you can talk to a MySQL server from Python without needing to first install the MySQL client libraries (which often requires compiling from source). # 2nd October 2009, 2:16 pm

TypePad Motion. Launched today at FOWA, Motion is a microblogging application written in Django that uses the TypePad API for all persistent storage—which means you can deploy it as server-side code on your own site, but scaling horizontally is handled by TypePad (you just need to scale out the state-free front end). # 1st October 2009, 3:51 pm

What’s New in Python 2.6 (via) Python 2.6 final has been released (the last 2.x version before 3.0). multiprocessing and simplejson (as json) are now in the standard library, any backwards compatible 3.0 features have been added and the official docs are now powered by Sphinx (used by Django 1.0 as well). There’s plenty more. # 2nd October 2008, 11:47 am

Using the New MySQL Query Profiler. Extremely powerful new feature in MySQL 5.0.37. Definitely something for the Django debug toolbar. # 1st October 2008, 1:20 am