DJP: A plugin system for Django
25th September 2024
DJP is a new plugin mechanism for Django, built on top of Pluggy. I announced the first version of DJP during my talk yesterday at DjangoCon US 2024, How to design and implement extensible software with plugins. I’ll post a full write-up of that talk once the video becomes available—this post describes DJP and how to use what I’ve built so far.
- Why plugins?
- Setting up DJP
- django-plugin-django-header
- django-plugin-blog
- django-plugin-database-url
- Writing a plugin
- Writing tests for plugins
- Why call it DJP?
- What’s next for DJP?
Why plugins?
Django already has a thriving ecosystem of third-party apps and extensions. What can a plugin system add here?
If you’ve ever installed a Django extension—such as django-debug-toolbar or django-extensions—you’ll be familiar with the process. You pip install
the package, then add it to your list of INSTALLED_APPS
in settings.py
—and often configure other picees, like adding something to MIDDLEWARE
or updating your urls.py
with new URL patterns.
This isn’t exactly a huge burden, but it’s added friction. It’s also the exact kind of thing plugin systems are designed to solve.
DJP addresses this. You configure DJP just once, and then any additional DJP-enabled plugins you pip install
can automatically register configure themselves within your Django project.
Setting up DJP
There are three steps to adding DJP to an existing Django project:
-
pip install djp
—or add it to yourrequirements.txt
or similar. -
Modify your
settings.py
to add these two lines:# Can be at the start of the file: import djp # This MUST be the last line: djp.settings(globals())
-
Modify your
urls.py
to contain the following:import djp urlpatterns = [ # Your existing URL patterns ] + djp.urlpatterns()
That’s everything. The djp.settings(globals())
line is a little bit of magic—it gives djp
an opportunity to make any changes it likes to your configured settings.
You can see what that does here. Short version: it adds "djp"
and any other apps from plugins to INSTALLED_APPS
, modifies MIDDLEWARE
for any plugins that need to do that and gives plugins a chance to modify any other settings they need to.
One of my personal rules of plugin system design is that you should never ship a plugin hook (a customization point) without releasing at least one plugin that uses it. This validates the design and provides executable documentation in the form of working code.
I’ve released three plugins for DJP so far.
django-plugin-django-header
django-plugin-django-header is a very simple initial example. It registers a Django middleware class that adds a Django-Composition:
HTTP header to every response with the name of a random Composition by Django Reinhardt (thanks,Wikipedia).
pip install django-plugin-django-header
Then try it out with curl
:
curl -I http://localhost:8000/
You should get back something like this:
...
Django-Composition: Nuages
...
I’m running this on my blog right now! Try this command to see it in action:
curl -I https://simonwillison.net/
The plugin is very simple. Its __init__.py registers middleware like this:
import djp @djp.hookimpl def middleware(): return [ "django_plugin_django_header.middleware.DjangoHeaderMiddleware" ]
That string references the middleware class in this file.
django-plugin-blog
django-plugin-blog is a much bigger example. It implements a full blog system for your Django application, with bundled models and templates and views and a URL configuration.
You’ll need to have configured auth and the Django admin already (those already there by default in the django-admin startproject
template). Now install the plugin:
pip install django-plugin-blog
And run migrations to create the new database tables:
python manage.py migrate
That’s all you need to do. Navigating to /blog/
will present the index page of the blog, including a link to a working Atom feed.
You can add entries and tags through the Django admin (configured for you by the plugin) and those will show up on /blog/
, get their own URLs at /blog/2024/<slug>/
and be included in the Atom feed, the /blog/archive/
list and the /blog/2024/
year-based index too.
The default design is very basic, but you can customize that by providing your own base template or providing custom templates for each of the pages. There are details on the templates in the README.
The blog implementation is directly adapted from my Building a blog in Django TIL.
The primary goal of this plugin is to demonstrate what a plugin with views, templates, models and a URL configuration looks like. Here’s the full __init__.py for the plugin:
from django.urls import path from django.conf import settings import djp @djp.hookimpl def installed_apps(): return ["django_plugin_blog"] @djp.hookimpl def urlpatterns(): from .views import index, entry, year, archive, tag, BlogFeed blog = getattr(settings, "DJANGO_PLUGIN_BLOG_URL_PREFIX", None) or "blog" return [ path(f"{blog}/", index, name="django_plugin_blog_index"), path(f"{blog}/<int:year>/<slug:slug>/", entry, name="django_plugin_blog_entry"), path(f"{blog}/archive/", archive, name="django_plugin_blog_archive"), path(f"{blog}/<int:year>/", year, name="django_plugin_blog_year"), path(f"{blog}/tag/<slug:slug>/", tag, name="django_plugin_blog_tag"), path(f"{blog}/feed/", BlogFeed(), name="django_plugin_blog_feed"), ]
It still only needs to implement two hooks: one to add django_plugin_blog
to the INSTALLED_APPS
list and another to add the necessary URL patterns to the project.
The from .views import ...
line is nested inside the urlpatterns()
hook because I was hitting circular import issues with those imports at the top of the module.
django-plugin-database-url
django-plugin-database-url is the smallest of my example plugins. It exists mainly to exercise the settings()
plugin hook, which allows plugins to further manipulate settings in any way they like.
Quoting the README:
Once installed, any
DATABASE_URL
environment variable will be automatically used to configure your Django database setting, using dj-database-url.
Here’s the full implementation of that plugin, most of which is copied straight from the dj-database-url documentation:
import djp import dj_database_url @djp.hookimpl def settings(current_settings): current_settings["DATABASES"]["default"] = dj_database_url.config( conn_max_age=600, conn_health_checks=True, )
If DJP gains traction, I expect that a lot of plugins will look like this—thin wrappers around existing libraries where the only added value is that they configure those libraries automatically once the plugin is installed.
Writing a plugin
A plugin is a Python package bundling a module that implements one or more of the DJP plugin hooks.
As I’ve shown above, the Python code for plugins can be very short. The larger challenge is correctly packaging and distributing the plugin—plugins are discovered using Entry Points which are defined in a pyproject.toml
file, and you need to get those exactly right for your plugin to be discovered.
DJP includes documentation on creating a plugin, but to make it as frictionless as possible I’ve released a new django-plugin cookiecutter template.
This means you can start a new plugin like this:
pip install cookiecutter
cookiecutter gh:simonw/django-plugin
Then answer the questions:
[1/6] plugin_name (): django-plugin-example
[2/6] description (): A simple example plugin
[3/6] hyphenated (django-plugin-example):
[4/6] underscored (django_plugin_example):
[5/6] github_username (): simonw
[6/6] author_name (): Simon Willison
And you’l get a django-plugin-example
directory with a fully configured plugin ready to be published to PyPI.
The template includes a .github/workflows
directory with actions that can run tests, and an action that publishes your plugin to PyPI any time you create a new release on GitHub.
I’ve used that pattern myself for hundreds of plugin projects for Datasette and LLM, so I’m confident this is an effective way to release plugins.
The workflows use PyPI’s Trusted Publishers mechanism (see my TIL), which means you don’t need to worry about API keys or PyPI credentials—configure the GitHub repo once using the PyPI UI and everything should just work.
Writing tests for plugins
Writing tests for plugins can be a little tricky, especially if they need to spin up a full Django environemnt in order to run the tests.
I previously published a TIL about that, showing how to have tests with their own tests/test_project
project that can be used by pytest-django.
I’ve baked that pattern into the simon/django-plugin
cookiecutter template as well, plus a single default test which checks that a hit to the /
index page returns a 200 status code—still a valuable default test since it confirms the plugin hasn’t broken everything!
The tests for django-plugin-django-header and for django-plugin-blog should provide a useful starting point for writing tests for your own plugins.
Why call it DJP?
Because django-plugins already existed on PyPI, and I like my three letter acronyms there!
What’s next for DJP?
I presented this at DjangoCon US 2024 yesterday afternoon. Initial response seemed positive, and I’m going to be attending the conference sprints on Thursday morning to see if anyone wants to write their own plugin or help extend the system further.
Is this a good idea? I think so. Plugins have been transformative for both Datasette and LLM, and I think Pluggy provides a mature, well-designed foundation for this kind of system.
I’m optimistic about plugins as a natural extension of Django’s existing ecosystem. Let’s see where this goes.
More recent articles
- OpenAI DevDay: Let’s build developer tools, not digital God - 2nd October 2024
- OpenAI DevDay 2024 live blog - 1st October 2024
- Weeknotes: Three podcasts, two trips and a new plugin system - 30th September 2024