CSV Upload

[simon-leech]

CSV Uploading

Description

This PR provides a new utility lib.utils.csvUpload. This module takes an input and parameters and stores the data to the database in chunks.
An updateQuery is ran after the importing is successful, which can be used to generate the geometry of rows for example.
The module returns a Promise, and an outcome object.
The outcome object will contain a success flag, of true or false indicating the success of the import, and a message, which can be displayed to the user in the plugins.csv_upload.

A new plugins.csv_upload has been added, this provides the UI for the CSV Upload - which is a button that opens a dialog for importing. Alerts are used when the Promise resolves to inform the user of the import.
This provides both a mapp.ui.layers.panels.csv_upload and mapp.ui.locations.entries.csv_upload.

GitHub Issue

#1676

Type of Change

Please delete options that are not relevant, and select all options that apply.

• ✅ New feature (non-breaking change which adds functionality)
• ✅ Documentation

How have you tested this?

Tested on bugs_testing/plugins/csv_upload/workspace.json

Testing Checklist

Please delete options that are not relevant, and select all options that apply.

• ✅ Existing Tests still pass
• ✅ Ran locally on my machine

Code Quality Checklist

Please delete options that are not relevant, and select all options that apply.

• ✅ My code follows the guidelines of XYZ
• ✅ My code has been commented
• ✅ Documentation has been updated
• ✅ New and existing unit tests pass locally with my changes
• ✅ Main has been merged into this PR

[AlexanderGeere]

The upload button on the entry doesn't seem to get the id, when I attempt the upload the id is undefined, and nothing goes into the table. Side note it does say the import was successful.

[dbauszus-glx]

This is way too complicated for general plugin / utility.

There is a situation where a very specific update query is executed from the resolution of multiple promises with insert quaeries which have to modules and must receive a body with an array property arr.

I would recommend to cleanup, test, and document POST queries with an array body first.

This is a valid select statement that will return to you the array sent as POST body.

    "select_body": {
      "template": "SELECT %{body}",
      "value_only": true
    },

Any rerences to the updateQuery should be removed from the csvUpload utility. These should be handled in the plugin which I would add in a seperate the PR.

The csvUpload utility should be in line with the csvDownload utility which merges multiple array responses into a CSV file.

The csvDownload method should be fully documented before another even more complex utility is added.

The specification for the csvUpload utility method should be to split a csv into seperate array bodies provided as POST body to a SQL query and return an array of responses from the individual queries once all queries are resolved. It must be possible to provide queryparams for the individual queries as well as a parameter for the payload to use in the array split. It should be possible to define whether all queries should be run in parallel or whether the queries should be sent consecutively.

This can be tested with very small csv file and providing a small enough payload param to force this csv to be split into at least two parts to be sent to the simple select %{body} statement which returns the body.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud