Skip to content

Latest commit

 

History

History
184 lines (119 loc) · 8.86 KB

RFC.md

File metadata and controls

184 lines (119 loc) · 8.86 KB
Raw

Hasura Modules: Request for Comments

TL;DR: Proof of Concept

Rationale

Recurrent patterns are present in databases and areas currently covered by Hasura, for example sending of emails, calendars, authentication, soft delete, payment, storage, audit trail... Whereas not all the business logic of such modules cansometimes be embbeded inside an Hasura instance, Hasura can be of great help to ease their implementation, through pieces of metadata and SQL schema.

Related

Features

A module is a set of sql migrations and metadata that can be installed, upgraded and uninstalled with predictable or no side-effects on an existing database managed by Hasura.

Installation

The module installation should run series of sql migrations initially generated by the Hasura console, in either config v1 or config v2:

hasura modules install <module-name>

Upgrade

hasura modules upgrade <module-name>

Modules should be able to be upgraded, meaning the underlying sql design elements can be altered or dropped, and new ones can be created.

The module metadata should be upgradable too, which implies an incremental metadata system that exists in Hasura console config v1, but not anymore on config v2.

Uninstallation

Unintall a module i.e. its Postgres structure and hasura metadata:

hasura modules uninstall <module-name> --remove-externals=<none|all|sql|metadata>

Some migrations in the module can be flagged as "external", meaning that they will possibly interacting direcly with the rest of the application, or that they should be kept even after uninstalling the module, for instance when some key data is stored in some of the installed tables.

Example: a user table in an authentication module, that is most likely going to be re-used in the rest of the application.

The developer can then specify what to do with such external migrations with the --remove-externals option. By default, externals would be set to 'all'.

List

hasura modules list

Module information

hasura modules show <module-name>

Public repository

A public repository that would contain the verified/recommended modules should be maintained, so they are available out of the box with the Hasura CLI. Community contributions should also be taken into account in a dedicated category. Here is a working example with two modules of what it would look like.

Examples

Example 1: Embedded Module

Some modules won't need any external service to be fully functionnal with Postgres and Hasura.

Let's imagine quickly a set of SQL functions, triggers and views that allow to alter the tables to implement a soft delete system. Some functions could be mapped as Hasura metadata so an admin user can add soft deletion to a given table. Soft deleted records would be moved to a dedicated table with their values stored as JSONB. Another function would be available to undo a deletion.

(Note: this way is not necessarily the best to implement soft deletion, and is described as a means to illustrate this RFC.)

Installation

  • Ensure Hasura is running correctly and that the project config.yaml is correctly configured
  • hasura modules install soft-delete
    • This would run SQL migrations that create the necessary tables, functions, etc in a dedicated soft_delete Postgres schema, and add the corresponding operations to the existing Hasura metadata.
    • The corresponding SQL migrations would be added to the migrations directory
    • The corresponding Hasura metadata would be added to the metadata directory if using config v2.

Upgrade

Let's say a new version of the soft-delete is available. It contains new SQL migrations as well as a adjustments in the metadata. The developer can then upgrade the module with hasura modules upgrade soft-delete.

Uninstall

hasura modules uninstall soft-delete

Example 2: Microservice Module

Let's imagine now a module that allows the sending of email through an external email service, the tracking of the mails sent by users, the ability to define templates... This module would require some tables, relationships and most likely event triggers or actions.

It would also require additional steps to follow one a module is installed, e.g.:

  • Create an account, fetch some credentials like an access token...
  • For a self-hosted microservice, for instance a docker container, pull, configure and run a docker image.
  • Add environment variables to Hasura, and restart the server to take them into account

All the above steps are hardly automatisable. However, a module should be able to print an explanation of the next steps forward, following its installation. For example, an Hasura action is internally defined with the url {{EMAIL_SERVICE_URL}}/send-email, and would require an account id EMAIL_SERVICE_ACCOUNT_ID and an access token EMAIL_SERVICE_ACCOUNT_TOKEN variable that would be sent to the handler as header. The post-install text would be:

In order to activate the module, you should add the following
environment variables in your Hasura GraphQL Engine server:

EMAIL_SERVICE_URL=https://hasura-emailer.io
EMAIL_SERVICE_ACCOUNT_ID=<your hasura emailer account id>
EMAIL_SERVICE_ACCOUNT_TOKEN=<your hasura emailer account token>

For further information, please visit https://docs.hasura-emailer.io

Proceedings

Interface

Initially the Hasura modules would be available through the Hasura CLI, but the inclusion of the following features to the console is to be considered.

Keep track of the SQL migrations and Hasura metadata

As a start, this feature could use the existing Hasura version numbering system. Although it is very unlikely two migrations would once get the same timestamp, we could consider adding a module column to the hdb_catalog.schema_migrations table.

Incremental metadata

It has been mentionned modules could be implemented in adding an hasura metadata import command to a config v2 project. While it would work well for module installation, it may become a problem when upgrading or uninstalling a module. If an upgrade changes the existing metadata of a module, a simple import would likely keep obsolete parts of the metadata, or create inconsistencies. In a similar way, a module removal will create challenges in the identification of the metadata elements to pull out of the project, and the ones to keep.

We could consider a CLI command that would generate the union of the intersection of metadata files from two separate versions. However module maintenance may become overcomplicated when the number of version increases.

The simplest way to proceed would therefore to use the existing config v1 incremental metadata system, and to accept the caveat of developing modules in config v1 while finding a more suitable solution.

External SQL/metadata

Some parts of the module sql/metadata may need to be optionnaly kept while uninstalling a module. Such parts should be isolated in granular migrations, and flagged in the configuration file of the module, so the user can decide to keep them or not when uninstalling a module.

Dependencies between modules

Some modules may eventually require other modules. The installation and update of a module could trigger a preliminary check of installed modules to the required version, and install them when needed. This process would be recursive.

Module Structure

module-name/
  config.yaml
    description:
    repo: http://website-or-repo.org
    install: # Optional
      pre: message shown at the beginning of the installation process
      post: message shown at the end of the installation process
    uninstall: # Optional
      pre: message shown at the beginning of the uninstallation process
      post: message shown at the end of the uninstallation process
    upgrade: # Optional
      pre: message shown at the beginning of the upgrade process
      post: message shown at the end of the upgrade process
    external: # Optional
      - <migration-number-A>
      - <migration-number-B>
    versions: # Optional
      - <version-tag>: <migration-number>
    dependencies: # Optional
      - module: <module-name>
        version: <version-tag> # Optional
  migrations/
    <migration-number>_module_<module-name>_migration_description
      up.sql
      up.yaml
      down.sql
      down.yaml

Prioritisation

Following MoSCoW:

  • Must
    1. list
    2. install
    3. uninstall
    4. upgrade
  • Should
    • User Interface through the console
    • Standard / third-party (contrib) modules
    • Modules can be written in both config v1 and config v2
    • 'Extended' sql and metadata
  • Could
    • dependencies between modules
  • Won't

Proof of Concept

A proof of concept is available here