Simon Willison’s Weblog

SQLModel. A new project by FastAPI creator Sebastián Ramírez: SQLModel builds on top of both SQLAlchemy and Sebastián’s Pydantic validation library to provide a new ORM that’s designed around Python 3’s optional typing. The real brilliance here is that a SQLModel subclass is simultaneously a valid SQLAlchemy ORM model AND a valid Pydantic validation model, saving on duplicate code by allowing the same class to be used both for form/API validation and for interacting with the database. # 24th August 2021, 11:16 pm

PugSQL. Interesting new twist on a definitely-not-an-ORM library for Python. With PugSQL you define SQL queries in files, give them names and then load them into a module which allows you to execute them as Python methods with keyword arguments. You can mark statements as only returning a single row (or a single scalar value) with a comment at the top of their file. # 3rd July 2019, 6:19 pm

Appending the request URL to SQL statements in Django. A clever frame-walking monkey-patch which pulls the most recent HttpRequest object out of the Python stack and adds the current request.path to each SQL query as an SQL comment, so you can see it in debugging tools such as slow query logs and the PostgreSQL “select * from pg_stat_activity” query. # 2nd June 2010, 9:09 am

Cache Machine: Automatic caching for your Django models. This is the third new ORM caching layer for Django I’ve seen in the past month! Cache Machine was developed for zamboni, the port of addons.mozilla.org to Django. Caching is enabled using a model mixin class (to hook up some post_delete hooks) and a custom caching manager. Invalidation works by maintaining a “flush list” of dependent cache entries for each object—this is currently stored in memcached and hence has potential race conditions, but a comment in the source code suggests that this could be solved by moving to redis. # 11th March 2010, 7:35 pm

Johnny Cache. Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully. # 28th February 2010, 10:55 pm

django-batch-select (via) A smart attempt at solving select_related for many-to-many relationships in Django. Add a custom manager to your model and call e.g. Entry.objects.all()[:10].batch_select(“tags”) to execute two queries—one pulling back the first ten entries and another using an “IN” query against the tags table to pull back all of the tags for those entries in one go. # 23rd November 2009, 4:19 pm

Django 1.2 planned features. The votes are in and the plan for Django 1.2 has taken shape - features are split in to high, medium and low priority. There's some really exciting stuff in there - outside of the things I've already talked about, I'm particularly excited about multidb, Model.objects.raw(SQL), the smarter {% if %} tag and class-based generic views. # 26th October 2009, 10:38 am

Django 1.1 release notes (via) Django 1.1 is out! Congratulations everyone who worked on this, it’s a fantastic release. New features include aggregate support in the ORM, proxy models, deferred fields and some really nice admin improvements. Oh, and the testing framework is now up to 10 times thanks to smart use of transactions. # 29th July 2009, 9:34 am

South’s Design. Andrew Godwin explains why South resorts to parsing your models.py file in order to construct information about for creating automatic migrations. # 13th May 2009, 12:30 pm

Haystack (via) A brand new modular search plugin for Django, by Daniel Lindsley. The interface is modelled after the Django ORM (complete with declarative classes for defining your search schema) and it ships with backends for both Solr and pure-python Whoosh, with more on the way. Excellent documentation. # 17th April 2009, 9:53 pm

DB2 support for Django is coming. From IBM, under the Apache 2.0 License. I’m not sure if this makes it hard to bundle it with the rest of Django, which uses the BSD license. # 18th February 2009, 10:58 pm

Secrets of the Django ORM. An undocumented (and unsupported) method of poking a Django QuerySet’s internal query to add group_by and having clauses to a SQL query. # 8th November 2008, 11:49 pm

Django 1.0 alpha release notes. The big features are newforms-admin, unicode everywhere, the queryset-refactor ORM improvements and auto-escaping in templates. # 22nd July 2008, 6:04 am

Queryset-refactor branch has been merged into trunk. Malcolm’s latest Django masterpiece is complete. # 27th April 2008, 7:21 am

mysql_cluster (via) My Russian isn’t all that good, but this looks like a neat way of getting Django to talk to a master/slave setup, written by Ivan Sagalaev. UPDATE: English docs are linked from the comments. # 21st March 2008, 8:45 am

Queryset Implementation. Malcolm explains the work that has gone in to the queryset-refactor branch. Executive summary: Python’s ORM is probably a lot better at SQL than you are. # 19th March 2008, 9:43 am

Caching Layer for Django ORM. Interesting extension to Django’s ORM that adds automatic caching of querysets and smart cache invalidation. # 23rd January 2008, 3:18 pm

Using the extra() QuerySet modifier in Django for WeGoEat. You can use select() on a QuerySet to obtain extra values using subqueries. # 24th October 2007, 7:28 pm

tranquil. Inspired take on the Django ORM to SQLAlchemy problem: lets you define your models with the Django ORM but use SQLAlchemy to run queries against them. # 9th October 2007, 2:30 am

DbMigration—a schema migration tool for Django. Nice and simple tool for adding schema migrations to a Django application. # 27th September 2007, 3:04 pm

django-sphinx (via) More code from Curse Gaming; this time a really nice API for adding Sphinx full-text search to a Django model. # 9th September 2007, 12:35 am

AuditTrail. Add change tracking and history to a Django model with a single line of code. Doesn’t handle relationships though, which is definitely the toughest part of this problem. # 15th August 2007, 1 pm

Finding Lookup Items that Are Not Used. How to do left outer joins (and other custom SQL) using the Django ORM. # 13th August 2007, 5:08 pm

A nice example of when to use reduce in python. As a shortcut for assembling a large OR query using the Django ORM. # 2nd August 2007, 11:51 pm

Undelete in Django. Inspired by the conversation about undo the other day, Nathan Ostgard created a simple solution based around custom managers and a trashed_at model field. # 20th July 2007, 6:54 pm

Storm. New Python ORM from Canonical, emphasising multiple database support, intelligent local cache invalidation and a thin layer over the underlying SQL. # 9th July 2007, 8:44 am