Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a registration backend that sends the form/submission variables as JSON to some API endpoint #4908

Open
9 of 12 tasks
sergei-maertens opened this issue Dec 12, 2024 · 4 comments · May be fixed by #4956
Open
9 of 12 tasks
Assignees
Labels
enhancement owner: venlo topic: registration waiting for approval An estimate was made but the stakeholder still needs to approve it.
Milestone

Comments

@sergei-maertens
Copy link
Member

sergei-maertens commented Dec 12, 2024

There is a desire for a "simple" registration backend/plugin which just dumps the submission/form variables with their values as a "flat" JSON structure and sends this data to an API endpoint for further processing. The goal is to keep this super simple and light weight, and let the post-processing be done on the receiving end (with an enterprise service bus or other technologies).

Requirements

Example

An example form with the following configuration:

  • Authentication via DigiD, capturing a BSN
  • First form step with text fields firstName and lastName
  • Second form step with a user upload attachment

Should lead to a JSON object being sent with the shape:

{
    "values":
    {
        "auth_bsn": "123456782",
        "firstName": "Donnie",
        "lastName": "Darko",
        "attachment": "<base64 encoded data>"
    },
    "schema":
    {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "type": "object",
        "properties":
        {
            "auth_bsn":
            {
                "type": "string",
                "pattern": "^\\d{9}$"
            },
            "firstName":
            {
                "type": "string"
            },
            "lastName":
            {
                "type": "string"
            },
            "attachment":
            {
                "type": "string",
                "contentEncoding": "base64"
            }
        },
        "required":
        [
            "auth_bsn",
            "firstName",
            "lastName"
        ],
        "additionalProperties": false
    }
}

Tasks

  • Define new registration plugin - it's probably wise to gate it behind a if settings.DEBUG since I don't expect this to be done for 3.0
  • Implement plugin configuration options:
    • Service to use
    • Endpoint to send data to (relative to service API root)
    • Form variables (including static variables!) to include
  • Add docker-compose mock service (flask app?) to simulate a receiving server
  • Implement sending the data to the configured service in the defined format, taking into account the configuration options
  • Add configuration checks to configuration overview
  • Ensure that we can generate an appropriate json-schema for each form component type
  • Add tests

Pointers

Existing code that can be useful:

  • service fetch
  • email registration plugin
  • objects API registration configuration/options
@sergei-maertens sergei-maertens added this to the Release 3.1.0 milestone Dec 12, 2024
@sergei-maertens sergei-maertens added waiting for approval An estimate was made but the stakeholder still needs to approve it. owner: venlo labels Dec 12, 2024
@viktorvanwijk viktorvanwijk moved this from Todo to In Progress in Development Dec 16, 2024
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
Plugin name did not make sense before
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
API endpoint is relative, name should reflect that
viktorvanwijk added a commit that referenced this issue Dec 18, 2024
@viktorvanwijk
Copy link
Contributor

viktorvanwijk commented Dec 18, 2024

  • Optionally, a JSON Schema definition of the submitted data is provided in the schema key

Does 'optionally' mean that the configuration options should also include a checkbox (for example) with which you can select this?

@sergei-maertens
Copy link
Member Author

  • Optionally, a JSON Schema definition of the submitted data is provided in the schema key

Does 'optionally' mean that the configuration options should also include a checkbox (for example) with which you can select this?

No - it's a wish to include this, but having the actual data is the minimum requirement for this ticket. Including the JSON Schema is desired, but not crucial and should likely be the final task of this ticket to implement :)

viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Plugin name did not make sense before
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
API endpoint is relative, name should reflect that
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
It should match the corresponding property of the serializer (which is 'service' and not 'service_select')
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Not relevant anymore or duplicates
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
Wrong place and doesn't look great
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
…n plugin

Need to include all form variables listed in the options to a values dictionary
viktorvanwijk added a commit that referenced this issue Dec 19, 2024
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
This will raise a KeyError if a form variable is not available in the submitted data nor in the static variables. This is better than passing is silently
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
…e service

The instructions on how to start it is now in line with the other docker compose services
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
The configuration options could be saved without specifying any form variables to include. This does not make much sense. Added a minimum length of 1 for the form variables list in the serializer (Thanks to Robin for the hint)
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
Will check the connection to the configured service
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
…n editor

The field 'formVariables' is read-only, so need an extra const to modify its value. Note, this didn't show up in storybook testing, only in manual functional testing
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
…lugin

Just as an example of what will be implemented in #4980
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
* Use VCR instead of mock now
* Added check on the response
* Added check on attachment encoding
viktorvanwijk added a commit that referenced this issue Jan 2, 2025
* Use VCR instead of mock now
* Add test for invalid response from API endpoint
viktorvanwijk added a commit that referenced this issue Jan 3, 2025
PR feedback: global config checks are not really useful for this plugin right now.
viktorvanwijk added a commit that referenced this issue Jan 3, 2025
A bit more descriptive and avoids possible naming conflicts
viktorvanwijk added a commit that referenced this issue Jan 3, 2025
viktorvanwijk added a commit that referenced this issue Jan 3, 2025
formData will be an empty object if we add a new object, which means the default values for the configuration fields are missing
viktorvanwijk added a commit that referenced this issue Jan 3, 2025
* Add required and noManageChildProps to ServiceSelect and FormVariableSelect because they are required and need to prevent some Field props being passed down to the children
* Remove max length of FormioVariableKeyField: it is based on a text field which has no length constraints
* Remove id field from RelativeAPIEndpoint TextInput: is managed automatically
* Remove Required from JSONDumpOptions: required is the default for TypedDict
* Add default value for relative_api_endpoint: guarantees that the key is always present, which gives a strict type definition
* Remove content type headers: is set automatically when using the json kwargs
* Replace 'Included/Not Included' text with icons: makes it easier for the user to see whether it is included.
viktorvanwijk added a commit that referenced this issue Jan 7, 2025
The requests.Response object is not JSON serializable, so need to just add the json of it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement owner: venlo topic: registration waiting for approval An estimate was made but the stakeholder still needs to approve it.
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

2 participants