Datasette plugins

Datasette provided a well-crafter example of building a Pragmatic, “God Object” based Plugin Architecture in an async environment.

While Dishka + Pluggy focuses on a “Purist” Dependency Injection approach (using Dishka), Datasette takes a different route: Context Passing. Instead of injecting specific services, it passes the entire Application instance (self) to almost every hook.

1. The “Async Bridge” Pattern (Sync/Async Hybridity)¶

Pluggy is inherently synchronous. Datasette is an ASGI (Async) application. To bridge this, Datasette implements a specific pattern to allow plugins to be either synchronous or asynchronous transparently.

The Mechanism (datasette/utils/__init__.py):
Datasette defines a wrapper called await_me_maybe.

async def await_me_maybe(value):
    "If value is callable, call it. If awaitable, await it. Otherwise return it."
    if callable(value):
        value = value()
    if asyncio.iscoroutine(value):
        value = await value
    return value

Usage (datasette/app.py):
When the core invokes a hook, it doesn’t just take the result. It iterates through the results and awaits them if necessary.

# From datasette/app.py
for hook in pm.hook.prepare_jinja2_environment(env=self._jinja_env, datasette=self):
    await await_me_maybe(hook)

Insight: You don’t need an async plugin framework to support async plugins. You can build an async execution layer on top of a synchronous registry like Pluggy.

2. The “God Object” Context Pattern¶

In the Dishka approach, we defined specific interfaces (Greeter, Database). In Datasette, the API is the application instance.

The Hook Spec (datasette/hookspecs.py):

@hookspec
def prepare_connection(conn, database, datasette):
    """Modify SQLite connection..."""

Every hook receives datasette. This is the Service Locator pattern implementation. * Pros: Drastically lowers the barrier to entry for plugin authors. They have access to everything (config, databases, renderers, actors) immediately via the datasette object. * Cons: High coupling. If the Datasette class changes its internal API, plugins might break. It makes unit testing plugins harder because you have to mock the whole Datasette object (seen in tests/plugins/my_plugin.py).

3. Composition of Logic via Data (The Permission System)¶

This is perhaps the most sophisticated part of Datasette’s architecture. Instead of plugins returning True/False for permissions (which requires loading data into Python), plugins return SQL fragments that are compiled into a single query.

The Code (datasette/utils/actions_sql.py):
The system builds a Common Table Expression (CTE) query.
1. Core: Defines the “Base” resources.
2. Plugins: Contribute SQL snippets via permission_resources_sql.
3. Execution: The DB engine evaluates the logic.

# From datasette/utils/actions_sql.py
rule_sqls.append(
    f"""
    SELECT parent, child, allow, reason, '{permission_sql.source}' AS source_plugin FROM (
        {permission_sql.sql}
    )
    """.strip()
)

Insight: For high-performance extension points, allow plugins to inject logic definitions (like SQL or ASTs) rather than executing the logic in the host language (Python). This avoids the “N+1” query problem in permission checks.

4. Frontend/Asset Injection¶

Datasette demonstrates how to make a backend Python app “Full Stack Extensible”. Plugins aren’t just backend logic; they can inject CSS, JS, and Menu items.

The Implementation:
1. Collection: The extra_css_urls and extra_js_urls hooks collect resources.
2. Rendering: These are passed to the Jinja2 template context in datasette/app.py.
3. Output: The base template (datasette/templates/base.html) iterates over these lists to render <link> and <script> tags.

<!-- datasette/templates/base.html -->
{% for url in extra_css_urls %}
    <link rel="stylesheet" href="{{ url.url }}">
{% endfor %}

Insight: A plugin system for a web app must bridge the gap between the backend registry and the frontend templates.

5. The “Canned Query” Pattern (Configuration as Code)¶

Datasette allows plugins to define “Canned Queries” (pre-written SQL). * Standard way: Define them in metadata.json (static configuration). * Plugin way: Implement the canned_queries hook.

Insight: Plugins can be used to generate dynamic configuration. Instead of reading a static config file, the app asks plugins “Do you have any configuration to add?” at runtime. This allows for dynamic routes and queries based on external factors (e.g., checking a different database to see what queries should be enabled).

6. Defensive Plugin Loading¶

Datasette wraps plugin execution heavily to prevent crashes.

Code (datasette/app.py):

try:
    pm.register(mod)
except ValueError:
    # Plugin already registered
    pass

And datasette/plugins.py allows loading plugins via an environment variable DATASETTE_LOAD_PLUGINS, bypassing standard discovery.

Insight: Real-world plugin systems need robust error handling around the loading mechanism. You cannot trust that entry points are valid or that plugins won’t conflict.

Summary: Datasette vs. The Dishka Approach¶

Feature	Datasette (Current Arch)	Dishka + Pluggy (Proposed Arch)
Access to Core	Service Locator: Passes `datasette` instance. High coupling, high convenience.	DI: Injects specific interfaces. Low coupling, higher boilerplate.
Async Handling	Manual: Wrappers iterate and `await` manually.	Native: The Container handles async provider resolution.
State Management	Plugins often mutate the `datasette` object or `request` object directly.	State is encapsulated in Scopes (Request/App) within the container.
Testing	Requires instantiating a full `Datasette` app object.	allows testing logic in isolation by mocking interfaces.

Conclusion:
Datasette’s architecture is optimized for Developer Experience (DX) and Ecosystem growth. By passing the “God Object” (datasette) to hooks, it ensures plugin authors rarely get stuck trying to access a specific utility. It trades strict architectural purity for pragmatic utility, which is likely why it has such a thriving plugin ecosystem.

Page last modified: 2025-11-26 13:50:23