The “await me maybe” pattern for Python asyncio
I’ve identified a pattern for handling potentially-asynchronous callback functions in Python which I’m calling the “await me maybe” pattern. It works by letting you return a value, a callable function that returns a value OR an awaitable function that returns that value.
Datasette also has a plugin system, built on top of Pluggy.
Pluggy is a beautifully designed mechanism for plugins. It works based on decorated functions, which are called at various points by Datasette itself.
from datasette import hookimpl @hookimpl def extra_js_urls(): return [ "https://code.jquery.com/jquery-3.5.1.min.js" ]
urls =  for url in pm.hook.extra_js_urls( template=template.name, datasette=datasette, ): urls.extend(url)
What’s up with the
datasette= parameters that are passed here?
Pluggy implements a form of dependency injection, where plugin hook functions can optionally list additional parameters that they would like to have access to.
The above simple example didn’t need any extra information. But imagine a plugin that only wants to inject jQuery on the
table.html template page:
@hookimpl def extra_js_urls(template): if template == "table.html": return [ "https://code.jquery.com/jquery-3.5.1.min.js" ]
Datasette actually provides several more optional argument for these plugin functions—see the plugin hooks documentation for full details.
What if we need to await something?
The datasette object that can be passed to plugin hooks is special: it provides an object that can be used for the following:
- Executing SQL against databases connected to Datasette
- Looking up Datasette metadata and configuration settings, including plugin configuration
- Rendering templates using the template environment configured by Datasette
- Performing checks against the Datasette permissions system
Here’s the problem: many of those methods on Datasette are awaitable—
await datasette.render_template(...) for example. But Pluggy is built around regular non-awaitable Python functions.
await inside a regular Python function.
That’s where the “await me maybe” pattern comes in.
The basic idea is that a function can return a value, OR a function-that-returns-a-value, OR an awaitable-function-that-returns-a-value.
If we want our
extra_js_urls(datasette) hook to execute a SQL query in order to decide what URLs to return, it can look like this:
@hookimpl def extra_js_urls(datasette): async def inner(): db = datasette.get_database() results = await db.execute("select url from js_files") return [r for r in results] return inner
Note that Python lets you define an
async def inner() function inside the body of a regular function, which is what we’re doing here.
The code that calls the plugin hook in Datasette can then look like this:
urls =  for url in pm.hook.extra_js_urls( template=template.name, datasette=datasette, ): if callable(url): url = url() if asyncio.iscoroutine(url): url = await url urls.append(url)
I use this pattern in a bunch of different places in Datasette, so today I refactored that into a utility function:
import asyncio async def await_me_maybe(value): if callable(value): value = value() if asyncio.iscoroutine(value): value = await value return value
body_scripts =  for extra_script in pm.hook.extra_body_script( template=template.name, database=context.get("database"), table=context.get("table"), columns=context.get("columns"), view_name=view_name, request=request, datasette=self, ): extra_script = await await_me_maybe(extra_script) body_scripts.append(Markup(extra_script))