1.0a8

This alpha release continues the migration of Datasette's configuration from metadata.yaml to the new datasette.yaml configuration file, introduces a new system for JavaScript plugins and adds several new plugin hooks.

See Datasette 1.0a8: JavaScript plugins, new plugin hooks and plugin configuration in datasette.yaml for an annotated version of these release notes.

Configuration

• Plugin configuration now lives in the datasette.yaml configuration file, passed to Datasette using the -c/--config option. Thanks, Alex Garcia. (#2093)

  datasette  -c  datasette.yaml

  Where datasette.yaml contains configuration that looks like this:

  plugins:
    datasette-cluster-map:
      latitude_column: xlat
      longitude_column: xlon

• Previously plugins were configured in metadata.yaml, which was confusing as plugin settings were unrelated to database and table metadata.

• The -s/--setting option can now be used to set plugin configuration as well. See Configuration via the command-line for details. (#2252)

  The above YAML configuration example using -s/--setting looks like this:

  datasette  mydatabase.db\
    -s  plugins.datasette-cluster-map.latitude_column  xlat\
    -s  plugins.datasette-cluster-map.longitude_column  xlon

• The new /-/config page shows the current instance configuration, after redacting keys that could contain sensitive data such as API keys or passwords. (#2254)

• Existing Datasette installations may already have configuration set in metadata.yaml that should be migrated to datasette.yaml. To avoid breaking these installations, Datasette will silently treat table configuration, plugin configuration and allow blocks in metadata as if they had been specified in configuration instead. (#2247) (#2248) (#2249)

Note that the datasette publish command has not yet been updated to accept a datasette.yaml configuration file. This will be addressed in #2195 but for the moment you can include those settings in metadata.yaml instead.

JavaScript plugins

Datasette now includes a JavaScript plugins mechanism, allowing JavaScript to customize Datasette in a way that can collaborate with other plugins.

This provides two initial hooks, with more to come in the future:

• makeAboveTablePanelConfigs() can add additional panels to the top of the table page.

• makeColumnActions() can add additional actions to the column menu.

Thanks Cameron Yick for contributing this feature. (#2052)

Plugin hooks

• New jinja2_environment_from_request(datasette, request, env) plugin hook, which can be used to customize the current Jinja environment based on the incoming request. This can be used to modify the template lookup path based on the incoming request hostname, among other things. (#2225)

• New family of template slot plugin hooks: top_homepage, top_database, top_table, top_row, top_query, top_canned_query. Plugins can use these to provide additional HTML to be injected at the top of the corresponding pages. (#1191)

• New track_event() mechanism for plugins to emit and receive events when certain events occur within Datasette. (#2240)

  • Plugins can register additional event classes using register_events(datasette).

  • They can then trigger those events with the datasette.track_event(event) internal method.

  • Plugins can subscribe to notifications of events using the track_event(datasette, event) plugin hook.

  • Datasette core now emits login, logout, create-token, create-table, drop-table, insert-rows, upsert-rows, update-row, delete-row events, documented here.

• New internal function for plugin authors: await db.execute_isolated_fn(fn), for creating a new SQLite connection, executing code and then closing that connection, all while preventing other code from writing to that particular database. This connection will not have the prepare_connection() plugin hook executed against it, allowing plugins to perform actions that might otherwise be blocked by existing connection configuration. (#2218)

Documentation

• Documentation describing how to write tests that use signed actor cookies using datasette.client.actor_cookie(). (#1830)

• Documentation on how to register a plugin for the duration of a test. (#2234)

• The configuration documentation now shows examples of both YAML and JSON for each setting.

Minor fixes

• Datasette no longer attempts to run SQL queries in parallel when rendering a table page, as this was leading to some rare crashing bugs. (#2189)

• Fixed warning: DeprecationWarning: pkg_resources is deprecated as an API (#2057)

• Fixed bug where ?_extra=columns parameter returned an incorrectly shaped response. (#2230)

0.64.6

• Fixed a bug where CSV export with expanded labels could fail if a foreign key reference did not correctly resolve. (#2214)

0.64.5

• Dropped dependency on click-default-group-wheel, which could cause a dependency conflict. (#2197)

1.0a7

• Fix for a crashing bug caused by viewing the table page for a named in-memory database. #2189

0.64.4

• Fix for a crashing bug caused by viewing the table page for a named in-memory database. #2189

1.0a6

• New plugin hook: actors_from_ids(datasette, actor_ids) and an internal method to accompany it, await .actors_from_ids(actor_ids). This mechanism is intended to be used by plugins that may need to display the actor who was responsible for something managed by that plugin: they can now resolve the recorded IDs of actors into the full actor objects. (#2181)
• DATASETTE_LOAD_PLUGINS environment variable for controlling which plugins are loaded by Datasette. (#2164)
• Datasette now checks if the user has permission to view a table linked to by a foreign key before turning that foreign key into a clickable link. (#2178)
• The execute-sql permission now implies that the actor can also view the database and instance. (#2169)
• Documentation describing a pattern for building plugins that themselves define further hooks for other plugins. (#1765)
• Datasette is now tested against the Python 3.12 preview. (#2175)

1.0a5

• When restrictions are applied to API tokens, those restrictions now behave slightly differently: applying the view-table restriction will imply the ability to view-database for the database containing that table, and both view-table and view-database will imply view-instance. Previously you needed to create a token with restrictions that explicitly listed view-instance and view-database and view-table in order to view a table without getting a permission denied error. (#2102)
• New datasette.yaml (or .json) configuration file, which can be specified using datasette -c path-to-file. The goal here to consolidate settings, plugin configuration, permissions, canned queries, and other Datasette configuration into a single single file, separate from metadata.yaml. The legacy settings.json config file used for Configuration directory mode has been removed, and datasette.yaml has a "settings" section where the same settings key/value pairs can be included. In the next future alpha release, more configuration such as plugins/permissions/canned queries will be moved to the datasette.yaml file. See #2093 for more details. Thanks, Alex Garcia.
• The -s/--setting option can now take dotted paths to nested settings. These will then be used to set or over-ride the same options as are present in the new configuration file. (#2156)
• New --actor '{"id": "json-goes-here"}' option for use with datasette --get to treat the simulated request as being made by a specific actor, see datasette --get. (#2153)
• The Datasette _internal database has had some changes. It no longer shows up in the datasette.databases list by default, and is now instead available to plugins using the datasette.get_internal_database(). Plugins are invited to use this as a private database to store configuration and settings and secrets that should not be made visible through the default Datasette interface. Users can pass the new --internal internal.db option to persist that internal database to disk. Thanks, Alex Garcia. (#2157).

1.0a4

This alpha fixes a security issue with the /-/api API explorer. On authenticated Datasette instances (instances protected using plugins such as datasette-auth-passwords) the API explorer interface could reveal the names of databases and tables within the protected instance. The data stored in those tables was not revealed.

For more information and workarounds, read the security advisory. The issue has been present in every previous alpha version of Datasette 1.0: versions 1.0a0, 1.0a1, 1.0a2 and 1.0a3.

Also in this alpha:

• The new datasette plugins --requirements option outputs a list of currently installed plugins in Python requirements.txt format, useful for duplicating that installation elsewhere. (#2133)
• Writable canned queries can now define a on_success_message_sql field in their configuration, containing a SQL query that should be executed upon successful completion of the write operation in order to generate a message to be shown to the user. (#2138)
• The automatically generated border color for a database is now shown in more places around the application. (#2119)
• Every instance of example shell script code in the documentation should now include a working copy button, free from additional syntax. (#2140)

1.0a3

This alpha release previews the updated design for Datasette's default JSON API. (#782)

The new default JSON representation for both table pages (/dbname/table.json) and arbitrary SQL queries (/dbname.json?sql=...) is now shaped like this:

{
  "ok": true,
  "rows": [
    {
      "id": 3,
      "name": "Detroit"
    },
    {
      "id": 2,
      "name": "Los Angeles"
    },
    {
      "id": 4,
      "name": "Memnonia"
    },
    {
      "id": 1,
      "name": "San Francisco"
    }
  ],
  "truncated": false
}

Tables will include an additional "next" key for pagination, which can be passed to ?_next= to fetch the next page of results.

The various ?_shape= options continue to work as before - see Different shapes for details.

A new ?_extra= mechanism is available for tables, but has not yet been stabilized or documented. Details on that are available in #262.

Smaller changes

• Datasette documentation now shows YAML examples for Metadata by default, with a tab interface for switching to JSON. (#1153)
• register_output_renderer(datasette) plugins now have access to error and truncated arguments, allowing them to display error messages and take into account truncated results. (#2130)
• render_cell() plugin hook now also supports an optional request argument. (#2007)
• New Justfile to support development workflows for Datasette using Just.
• datasette.render_template() can now accepts a datasette.views.Context subclass as an alternative to a dictionary. (#2127)
• datasette install -e path option for editable installations, useful while developing plugins. (#2106)
• When started with the --cors option Datasette now serves an Access-Control-Max-Age: 3600 header, ensuring CORS OPTIONS requests are repeated no more than once an hour. (#2079)
• Fixed a bug where the _internal database could display None instead of null for in-memory databases. (#1970)

0.64.3

• Added pip and setuptools as explicit dependencies. This fixes a bug where Datasette could not be installed using Rye. (#2065)