Simon Willison’s Weblog

486 items tagged “django”

Django Bakery (via) “A set of helpers for baking your Django site out as flat files”. Released by the LA Times Data Desk, who use it for a large number of projects from election results to data journalism interactives. Statically publishing these projects to S3 lets them handle huge traffic spikes at a very low cost. # 16th June 2018, 1:49 am

New in Django 2.0: Database instrumentation. I missed this previously. Django 2.0 shipped with one of my most-wanted features: the ability to easily instrument database calls (for logging and metrics) without having to monkey-patch or run an entirely new database backend. Can’t wait to try this out. # 21st May 2018, 9:28 pm

Django #8936: Add view (read-only) permission to admin (closed). Opened 10 years ago. Closed 15 hours ago. I apparently filed this issue during the first DjangoCon back in September 2008, when Adrian and Jacob mentioned on-stage that they would like to see a read-only permission for the Django Admin. Thanks to Olivier Dalang from Fiji and Petr Dlouhý from Prague it’s going to be a feature shipping in Django 2.1. Open source is a beautiful thing. # 17th May 2018, 1:40 pm

mendoza-trees-workshop (via) Eventbrite Argentina has an academy program to train new Python/Django developers. I presented a workshop there this morning showing how Django and Jupyter can be used together to iterate on a project. Since the session was primarily about demonstrating Jupyter it was mostly live-coding, but the joy of Jupyter is that at the end of a workshop you can go back and add inline commentary to the notebooks that you used. In putting together the workshop I learned about the django_extensions “/manage.py shell_plus --notebook” command—it’s brilliant! It launches Jupyter in a way that lets youdirectly import your Django models without having to mess around with DJANGO_SETTINGS_MODULE. # 8th May 2018, 5:22 pm

Describing events in code (via) Phil Gyford built an online directory of every play, movie, gig and exhibition he has been to in the past 38 years using a combination of digital archaeology and saved ticket stubs. He built it using Django and published this piece extensively describing the process he went through to design the data model. # 28th March 2018, 3:41 pm

Building a combined stream of recent additions using the Django ORM

I’m a big believer in the importance of a “recent additions” feed. Any time you’re building an application that involves users adding and editing records it’s useful to have a page somewhere that shows the most recent objects that have been created across multiple different types of data.

[... 1647 words]

The key to using Wagtail effectively is to recognise that there are multiple roles involved in creating a website: the content author, site administrator, developer and designer. These may well be different people, but they don’t have to be—if you’re using Wagtail to build your personal blog, you’ll probably find yourself hopping between those different roles. Either way, it’s important to be aware of which of those hats you’re wearing at any moment, and to use the right tools for that job. A content author or site administrator will do the bulk of their work through the Wagtail admin interface; a developer or designer will spend most of their time writing Python, HTML or CSS code. This is a good thing: Wagtail isn’t designed to replace the job of programming. Maybe one day someone will come up with a drag-and-drop UI for building websites that’s as powerful as writing code, but Wagtail is not that tool, and does not try to be.

The Zen of Wagtail # 1st March 2018, 4:10 pm

Wagtail 2.0. The leading Django content management system just released it’s 2.0 version—now Django 2.0 and Python 3 only, and with a new rich text editor based on Draft.js. I really like Wagtail—it’s full-feature,d mature, well-documented and philosophically aligned with how I think a CMS should work. I also like this line about the new Python 3 requirement: “Call us reckless neophiles, but we think that, nine years in, Python 3 is looking pretty solid.” # 1st March 2018, 4:06 pm

Conditional aggregation in Django 2.0 (via) I hadn’t realised how clever this new Django ORM feature by Tom Forbes is. It lets you build an aggregation against a subset of rows, e.g. Client.objects.aggregate(regular=Count(’pk’, filter=Q(account_type=Client.REGULAR)))—then if you are using PostgreSQL it translates it into a fast FILTER WHERE clause, while other databases emulate the same behaviour using a CASE statement. # 3rd February 2018, 9:38 pm

Channels 2.0. Andrew just shipped Channels 2.0—a major rewrite and redesign of the Channels project he started back in 2014. Channels brings async to Django, providing a logical, standardized way of supporting things like WebSockets and asynchronous execution on top of a Django application. Previously it required you to run a separate Twisted server and redis/RabbitMQ queue, but thanks to Python 3 async everything can now be deployed as a single process. And the new ASGI spec means its turtles all the way down! Everything from URL routing to view functions to middleware can be composed together using the same ASGI interface. # 2nd February 2018, 6:19 pm

Using setup.py in Your (Django) Project. Includes this neat trick: if you list manage.py in the setup(scripts=) argument you can call it from e.g. cron using the full path to manage.py within your virtual environment and it will execute in the correct context without needing to explicitly activate the environment first. # 2nd February 2018, 12:33 pm

django-postgres-copy (via) Really neat Django queryset add-on which exposes the PostgreSQL COPY statement for importing (and exporting) CSV data. MyModel.objects.from_csv(“filename.csv”). Built by the team of data journalists at the California Civic Data Coalition. # 26th January 2018, 12:43 am

GaretJax/django-click (via) I’ve been using Click to write command-line tools in Python recently (big datasette and csvs-to-sqlite use it) and its a delightful way of composing simple and complex CLI interfaces. I’ve always found Django’s default management command syntax hard to fit in my head—django-click means I can combine the two. # 19th January 2018, 11:19 pm

Django 2.0 released. The first version of Django to drop support for Python 2. I’ve been running the RC on my blog for the past 5 weeks and greatly enjoying the new mobile-optimized Django admin for posting links and quotations from my phone. The new simplified URL routing syntax (an optional alternative to regular expressions) is a very welcome improvement. # 2nd December 2017, 4:49 pm

pythondotorg/admin.py (via) There are some neat tricks in the Django application that powers Python.org (built a few years ago by RevSys). Here’s how their admin app handles creator/last_modified_by user relationships. # 19th November 2017, 6:28 am

DSF calls for applicants for a Django Fellow. This is a fantastic opportunity: the Django Software Foundation’s fellowship program is providing the opportunity for a new developer to get paid to work on Django 20-40 hours a week. Tim Graham has held this position full-time for the past three years and is looking to scale back to part-time, hence this new opportunity. Applications are due by December 18, 2017. # 16th November 2017, 10:31 pm

django-multitenant (via) Absolutely fascinating Django library for horizontally sharding a database using a multi-tenant pattern, from the team at Citus. In this pattern every relevant table includes a “tenant_id”, and all queries should specifically select against that ID. Once you have that in place, you can shard your rows across multiple different databases and route to the correct database based on the tenant ID, safe in the knowledge that joins will still work provided they are against other rows belonging to the same tenant. # 16th November 2017, 9:12 pm

Using “import refs” to iteratively import data into Django

I’ve been writing a few scripts to backfill my blog with content I originally posted elsewhere. So far I’ve imported answers I posted on Quora (background), answers I posted on Ask MetaFilter and content I recovered from the Internet Archive.

[... 560 words]

Squeezing every drop of performance out of a Django app on Heroku. Ben Firshman describes some lesser known tricks for scaling Django on Heroku—in particular, using gunicorn gevent asynchronous workers and setting up PostgreSQL connection pooling using django-db-geventpool. # 31st October 2017, 2:08 pm

arxiv-vanity (via) Beautiful new project from Ben Firshman and Andreas Jansson: “Arxiv Vanity renders academic papers from Arxiv as responsive web pages so you don’t have to squint at a PDF”. It works by pulling the raw LaTeX source code from Arxiv and rendering it to HTML using a heavily customized Pandoc workflow. The real fun is in the architecture: it’s a Django app running on Heroku which fires up on-demand Hyper.sh Docker containers for each individual rendering job. # 25th October 2017, 8:06 pm

A Brief Intro to Docker for Djangonauts (via) This is great—a really clear introduction to both Docker and Docker Compose, aimed at Django developers. Includes line-by-line annotations of an example Dockerfile and docker-compose.yml. # 18th October 2017, 9:06 pm

How to set up world-class continuous deployment using free hosted tools

I’m going to describe a way to put together a world-class continuous deployment infrastructure for your side-project without spending any money.

[... 1273 words]

My essential django package list. Insightful list of Django packages—many of which I hadn’t seen before—by Serafeim Papastefanos, each with a handy explanation of what it’s useful for and why. # 11th October 2017, 2:42 pm

Generate dynamic titles for /search/ pages. Fun little enhancement to my faceted search implementation: I now generate dynamic titles for each search results page describing the search, e.g. “Blogmarks tagged security in Feb, 2005” or ““python” in quotations tagged ruby, python in 2007”. # 8th October 2017, 7:47 pm

Building a statistical profiler in python. Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful. # 5th October 2017, 3:44 pm

Implementing faceted search with Django and PostgreSQL

I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter by content type (entries vs blogmarks vs quotation) and any combination of the above. Some example searches:

[... 3048 words]

Running gunicorn behind nginx on Heroku for buffering and logging

Heroku’s default setup for Django uses the gunicorn application server. Each Heroku dyno can only run a limited number of gunicorn workers, which means a limited number of requests can be served in parallel (around 4 per dyno is a good rule of thumb).

[... 400 words]

Why is snapEDA.com slow? Is it because it uses Django?

No, it’s not slow because it uses Django.

[... 36 words]

Are traditional web frameworks and languages like RubyOnRail, Spring Boot and PHP dying now when new fast reactive pure JavaScript frameworks and services like Meteor, Node, Angular 2.0 and Firebase are breaking ground?

No.

[... 40 words]

Should I use Django forms or pure HTML in order to do not establish borders for the growth of my app?

Use Django forms. Django scales horizontally on the front-end, so if your site needs to handle large amounts of traffic you just need to run multiple front-end servers—your form handling code will scale up just fine.

[... 65 words]