Simon Willison’s Weblog

Items tagged django in Feb

Filters: Month: Feb × django ×


Trying to end the pandemic a little earlier with VaccinateCA

This week I got involved with the VaccinateCA effort. We are trying to end the pandemic a little earlier, by building the most accurate database possible of vaccination locations and availability in California.

[... 1154 words]

django-zombodb (via) The hardest part of working with an external search engine like Elasticsearch is always keeping that index synchronized with your relational database. ZomboDB is a PostgreSQL extension which lets you create a new type of index backed by an external Elasticsearch cluster. Updated rows will be pushed to the index automatically, and custom SQL syntax can then be used to execute searches. django-zombodb is a brand new library by Flávio Juvenal which integrates ZomboDB directly into the Django ORM, letting you add Elasticsearch-backed functionality with just a few lines of extra configuration. It even includes custom Django migrations for enabling the extension in PostgreSQL! # 13th February 2019, 10:14 pm

Conditional aggregation in Django 2.0 (via) I hadn’t realised how clever this new Django ORM feature by Tom Forbes is. It lets you build an aggregation against a subset of rows, e.g. Client.objects.aggregate(regular=Count(’pk’, filter=Q(account_type=Client.REGULAR)))—then if you are using PostgreSQL it translates it into a fast FILTER WHERE clause, while other databases emulate the same behaviour using a CASE statement. # 3rd February 2018, 9:38 pm

Channels 2.0. Andrew just shipped Channels 2.0—a major rewrite and redesign of the Channels project he started back in 2014. Channels brings async to Django, providing a logical, standardized way of supporting things like WebSockets and asynchronous execution on top of a Django application. Previously it required you to run a separate Twisted server and redis/RabbitMQ queue, but thanks to Python 3 async everything can now be deployed as a single process. And the new ASGI spec means its turtles all the way down! Everything from URL routing to view functions to middleware can be composed together using the same ASGI interface. # 2nd February 2018, 6:19 pm

Using setup.py in Your (Django) Project. Includes this neat trick: if you list manage.py in the setup(scripts=) argument you can call it from e.g. cron using the full path to manage.py within your virtual environment and it will execute in the correct context without needing to explicitly activate the environment first. # 2nd February 2018, 12:33 pm

Should I use Django forms or pure HTML in order to do not establish borders for the growth of my app?

Use Django forms. Django scales horizontally on the front-end, so if your site needs to handle large amounts of traffic you just need to run multiple front-end servers—your form handling code will scale up just fine.

[... 65 words]

What web programming framework best supports ’drag and drop’ actions?  Please give examples of sites and/or plug-ins that support the interaction.

Drag and drop is a client-side thing—it has nothing to do with the server-side technology being used.

[... 72 words]

Python Django load MySQL database from csv files performance issue?

Don’t use the Django ORM for bulk imports—the performance overhead is pretty small for regular web page stuff, but it adds up if you are running millions of inserts.

[... 63 words]

How can I look up Django functions?

You can use the ./manage.py shell command to get a shell which will import any Django modules (or any of your own code) without complaining about the location of the settings.py module. Install IPython first to get a much more useful interactive shell when you run that command.

[... 190 words]

How can I install Django in a server without shell access?

I don’t think you can.

[... 42 words]

CSRF: Flash + 307 redirect = Game Over. Here’s the exploit that Django and Rails both just released fixes for. It’s actually a flaw in the Flash player. Flash isn’t meant to be able to make cross-domain HTTP requests with custom HTTP headers unless the crossdomain.xml file on the other domain allows them to, but it turns out a 307 redirect (like a 302, but allows POST data to be forwarded) confuses the Flash player in to not checking the crossdomain.xml on the host it is being redirect to. # 10th February 2011, 10:07 pm

Is South the best tool to use when doing database migrations in Django?

Yes. And I say that as an author of another Django migrations tool (dmigrations) which offered a small subset of South’s current functionality.

[... 42 words]

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. # 28th February 2010, 10:55 pm

Django Advent: Scaling Django. Mike Malone’s advice on scaling Django applications, including taking advantage of new features in 1.2. # 26th February 2010, 7:22 pm

jacobian’s django-deployment-workshop. Notes and resources from Jacob’s 3 hour Django deployment workshop at PyCon, including example configuration files for Apache2 + mod_wsgi, nginx, PostgreSQL and pgpool. # 19th February 2010, 2:28 pm

Django Advent. I can’t believe I haven’t linked to this already—Django Advent is “a series of articles about upcoming releases of the Django web framework”. Seven have been posted so far, covering topics from 1.2 including multi-db, messages, object permissions and natural keys. # 16th February 2010, 4:06 pm

Why toppcloud will not be agnostic. Ian Bicking’s toppcloud aims to offer deployment with the ease of use of AppEngine against a standard, open source Ubuntu + Python 2.6 + mod_wsgi + Varnish stack. Here he explains why he’s not going to vary the required components: keeping everything completely standardised means everyone gets the same bugs (and the same fixes). # 12th February 2010, 9:21 am

Presenting django-devserver, a better runserver. I really like this—it’s a Django management command (./manage.py rundevserver) which adds SQL logging and cache access summaries to the console output of Django’s local development server. It solves a similar set of problems to the debug toolbar, but requires slightly less setup and doesn’t inject additional HTML in to your pages. You can add your own custom modules to it as well. # 10th February 2010, 11:33 am

Integrate Tornado in Django. A handy ./manage.py runtornado management command for firing up a Tornado server that serves your Django application. # 8th February 2010, 11:12 am

django-springsteen and Distributed Search. Will Larson’s Django search library currently just talks to Yahoo! BOSS, but is designed to be extensible for other external search services. Interestingly, it uses threads to fire off several HTTP requests in parallel from within the Django view. # 25th February 2009, 10:28 pm

FAPWS3-0.2 (WSGI server based on libev). Another strong contender for Python’s answer to Mongrel—3500 requests/s for static files, 43 for a simple dynamic (Django powered) pages and 4.8 for a heavy SQL query—all benchmarked with 300 concurrent requests. # 25th February 2009, 10:21 pm

DB2 support for Django is coming. From IBM, under the Apache 2.0 License. I’m not sure if this makes it hard to bundle it with the rest of Django, which uses the BSD license. # 18th February 2009, 10:58 pm

Announcing django-viewtools. A really excellent idea—run ./manage.py viewtools --pdb /path/on/site/ to debug a view in your Django project that is raising an error using the Python debugger, or use --profile to run the full request cycle for that URL through the profiler. # 17th February 2009, 9:35 pm

The Django and Ubuntu Intrepid Almanac. Will Larson’s impressively comprehensive guide to configuring and securing an Ubuntu VPS from scratch to run Django, using PostgreSQL and Apache/mod_wsgi behind nginx. # 14th February 2009, 3:42 pm

EuroDjangoCon 2009. Tickets are now on sale for the conference, scheduled for 4th-6th of May (not March as I originally said) in Prague (followed by two days of development sprints). # 12th February 2009, 4:59 pm

Django Settings Tip—Setting Relative Paths. This is the first thing I do in every single one of my Django projects—it makes projects relocatable to other machines with just a couple of lines of code. I wouldn’t be at all upset to see it added to the default Django settings.py file created by ./manage.py startproject # 12th February 2009, 12:30 pm

solango. Another attempt at a Django/Solr integration library, based on code written for “a top 20 newspaper site” (I’d love to know which one). This is well documented, uses a registration model clearly inspired by the Django admin which keeps search related metadata out of your regular models and includes management commands for re-indexing and generating Solr schema.xml files. # 4th February 2009, 12:22 pm

queryset-refactor changeset 7126. Malcolm just checked model inheritance in to the queryset-refactor branch, with full documentation and unit tests. People have been requesting this for ages. # 18th February 2008, 9:46 am

themaneater.com Launch. The Maneater’s online edition is where Adrian cut his web development teeth, so it’s great to see them up and running on Django. Important to note that KeepAlive can completely murder Apache/Django performance. # 10th February 2008, 8:10 am

mydeco. Major new Django-powered start-up has launched, based in London. # 8th February 2008, 11:43 am