-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request 'Release v24.07' (#5) from release_24.07 into master
- Loading branch information
Showing
7 changed files
with
229 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,45 +1,39 @@ | ||
| Goobi workflow Plugin: goobi-plugin-step-download-and-verify-assets | ||
| =========================================================================== | ||
|
|
||
| <img src="https://goobi.io/wp-content/uploads/logo_goobi_plugin.png" align="right" style="margin:0 0 20px 20px;" alt="Plugin for Goobi workflow" width="175" height="109"> | ||
|
|
||
| This Step plugin for Goobi workflow allows to download assets and to verify the downloaded content from external pathes. | ||
|
|
||
| This is a plugin for Goobi workflow, the open source workflow tracking software for digitisation projects. More information about Goobi workflow is available under https://goobi.io. If you want to get in touch with the user community simply go to https://community.goobi.io. | ||
|
|
||
|
|
||
| Plugin details | ||
| --------------------------------------------------------------------------- | ||
|
|
||
| More information about the functionality of this plugin and the complete documentation can be found in the central documentation area at https://docs.goobi.io | ||
|
|
||
| Detail | Description | ||
| --------------------------- | ------------------------------- | ||
| **Plugin identifier** | intranda_step_download_and_verify_assets | ||
| **Plugin type** | Step Plugin | ||
| **Licence** | GPL 2.0 or newer | ||
| **Documentation (German)** | - no documentation available - | ||
| **Documentation (English)** | - no documentation available - | ||
|
|
||
|
|
||
| Goobi details | ||
| --------------------------------------------------------------------------- | ||
| Goobi workflow is an open source web application to manage small and large digitisation projects mostly in cultural heritage institutions all around the world. More information about Goobi can be found here: | ||
|
|
||
| Detail | Description | ||
| ------------------- | -------------------------- | ||
| **Goobi web site** | https://www.goobi.io | ||
| **Twitter** | https://twitter.com/goobi | ||
| **Goobi community** | https://community.goobi.io | ||
|
|
||
|
|
||
| Development | ||
| --------------------------------------------------------------------------- | ||
| This plugin was developed by intranda. If you have any issues, feedback, question or if you are looking for more information about Goobi workflow, Goobi viewer and all our other developments that are used in digitisation projects please get in touch with us. | ||
|
|
||
| Contact | Details | ||
| ----------------- | ---------------------------------------------------- | ||
| **Company name** | intranda GmbH | ||
| **Address** | Bertha-von-Suttner-Str. 9, 37085 Göttingen, Germany | ||
| **Web site** | https://www.intranda.com | ||
| **Twitter** | https://twitter.com/intranda | ||
| # Goobi workflow Plugin: goobi-plugin-step-download-and-verify-assets | ||
|
|
||
| <img src="https://goobi.io/wp-content/uploads/logo_goobi_plugin.png" align="right" style="margin:0 0 20px 20px;" alt="Plugin for Goobi workflow" width="175" height="109"> | ||
|
|
||
| This Step plugin for Goobi workflow allows to download assets and to verify the downloaded content from external paths. | ||
|
|
||
| This is a plugin for Goobi workflow, the open source workflow tracking software for digitisation projects. More information about Goobi workflow is available under https://goobi.io. If you want to get in touch with the user community simply go to https://community.goobi.io. | ||
|
|
||
| ## Plugin details | ||
|
|
||
| More information about the functionality of this plugin and the complete documentation can be found in the central documentation area at https://docs.goobi.io | ||
|
|
||
| Detail | Description | ||
| --------------------------- | ---------------------- | ||
| **Plugin identifier** | intranda_step_download_and_verify_assets | ||
| **Plugin type** | step | ||
| **Licence** | GPL 2.0 or newer | ||
| **Documentation (German)** | https://docs.goobi.io/workflow-plugins/v/eng/step/goobi-plugin-step-download-and-verify-assets | ||
| **Documentation (English)** | https://docs.goobi.io/workflow-plugins/v/ger/step/goobi-plugin-step-download-and-verify-assets | ||
|
|
||
| ## Goobi details | ||
|
|
||
| Goobi workflow is an open source web application to manage small and large digitisation projects mostly in cultural heritage institutions all around the world. More information about Goobi can be found here: | ||
|
|
||
| Detail | Description | ||
| --------------------------- | --------------------------- | ||
| **Goobi web site** | https://www.goobi.io | ||
| **Goobi community** | https://community.goobi.io | ||
| **Goobi documentation** | https://docs.goobi.io | ||
|
|
||
| ## Development | ||
|
|
||
| This plugin was developed by intranda. If you have any issues, feedback, question or if you are looking for more information about Goobi workflow, Goobi viewer and all our other developments that are used in digitisation projects please get in touch with us. | ||
|
|
||
| Contact | Details | ||
| --------------------------- | ---------------------------------------------------- | ||
| **Company name** | intranda GmbH | ||
| **Address** | Bertha-von-Suttner-Str. 9, 37085 Göttingen, Germany | ||
| **Web site** | https://www.intranda.com |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| --- | ||
| title: Download und Verifizieren von Dateien | ||
| identifier: intranda_step_download_and_verify_assets | ||
| published: true | ||
| description: Dieses Step-Plugin ermöglicht es, mehrere Dateien von URLs herunterzuladen und mit Checksummen zu verifizieren, die als Vorgangseigenschaften festgelegt wurden. Rückmeldungen über den Verlauf können zu verschiedenen Systemen oder auch lokal innerhalb des Journals gespeichert werden. | ||
| --- | ||
| ## Einführung | ||
| Dieses Plugin liest URLs bzw. Hash-Werte aus mehreren konfigurierten Vorgangseigenschaften ein, lädt die Dateien von der definierten URL herunter und verglicht sie anschließend mit dem zugehörigen Hash-Wert. Abschließend können mehrere Rückmeldungen gegeben werden, je nachdem ob der Status `success` oder `error` lautet. Diese Rückmeldungen können per REST zu einem anderen System geschickt oder einfach innerhalb des Journals geloggt werden. | ||
|
|
||
|
|
||
| ## Installation | ||
| Zur Installation des Plugins muss die folgende Datei installiert werden: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/plugins/step/plugin_intranda_step_download_and_verify_assets-base.jar | ||
| ``` | ||
|
|
||
| Die Konfigurationsdatei befindet sich üblicherweise hier: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/config/plugin_intranda_step_download_and_verify_assets.xml | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ## Konfiguration | ||
| Der Inhalt dieser Konfigurationsdatei sieht beispielhaft wie folgt aus: | ||
|
|
||
| ```xml | ||
| <config_plugin> | ||
| <!-- | ||
| order of configuration is: | ||
| 1.) project name and step name matches | ||
| 2.) step name matches and project is * | ||
| 3.) project name matches and step name is * | ||
| 4.) project name and step name are * | ||
| --> | ||
|
|
||
| <config> | ||
| <!-- which projects to use for (can be more then one, otherwise use *) --> | ||
| <project>*</project> | ||
| <step>*</step> | ||
|
|
||
| <!-- Configure here how many times shall be maximally tried before reporting final results. OPTIONAL. DEFAULT 1. --> | ||
| <maxTryTimes>3</maxTryTimes> | ||
|
|
||
| <!-- This tag accepts the following three attributes: | ||
| - @urlProperty: name of the property that holds the URL of the file | ||
| - @hashProperty: name of the property that holds the checksum of the file | ||
| - @folder: configured name of the target folder that shall be used to download the file. OPTIONAL. DEFAULT master. | ||
| --> | ||
| <fileNameProperty urlProperty="DraftUri" hashProperty="DraftHash" folder="master" /> | ||
| <fileNameProperty urlProperty="AssetUriSplitted" hashProperty="AssetHashSplitted" folder="master" /> | ||
|
|
||
| <!-- A response tag accepts four attributes: | ||
| - @type: success | error. Determines by which cases this configured response shall be activated. | ||
| - @method: OPTIONAL. If not configured or configured blankly, then the response will be performed via journal logs. Non-blank configuration options are: put | post | patch. | ||
| - @url: URL to the target system expecting this response. MANDATORY if @method is not blank.. | ||
| - @message: Message that shall be logged into journal. ONLY needed when @method is blank. | ||
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | ||
| One can also define a JSON string inside a pair of these tags, which will be used as JSON body to shoot a REST request. | ||
| --> | ||
| <!-- Usage of Goobi variables in @url as well as @message is allowed. --> | ||
| <response type="success" method="put" url="URL_ZU_BACH/upload_successful/{meta.ThesisId}" /> | ||
|
|
||
| <!-- For error cases there is no need for a response back to BACH, but an error message should be logged into journal. --> | ||
| <!-- Log ERROR_MESSAGE into journal as a signal of errors --> | ||
| <response type="error" message="ERROR_MESSAGE" /> | ||
|
|
||
| <!-- Example for REST calls with json body --> | ||
| <!-- | ||
| <response type="success" method="put" url="CHANGE_ME"> | ||
| { | ||
| "id": 0, | ||
| "name": "string", | ||
| "value": "string" | ||
| } | ||
| </response> | ||
| --> | ||
|
|
||
| </config> | ||
|
|
||
| </config_plugin> | ||
| ``` | ||
|
|
||
| Der Block `<config>` kann für verschiedene Projekte oder Arbeitsschritte wiederholt vorkommen, um innerhalb verschiedener Workflows unterschiedliche Aktionen durchführen zu können. | ||
|
|
||
| | Wert | Beschreibung | | ||
| | :--- | :--- | | ||
| | `project` | Dieser Parameter legt fest, für welches Projekt der aktuelle Block `<config>` gelten soll. Verwendet wird hierbei der Name des Projektes. Dieser Parameter kann mehrfach pro `<config>` Block vorkommen. | | ||
| | `step` | Dieser Parameter steuert, für welche Arbeitsschritte der Block `<config>` gelten soll. Verwendet wird hier der Name des Arbeitsschritts. Dieser Parameter kann mehrfach pro `<config>` Block vorkommen. | | ||
| | `maxTryTimes` | Dieser Wert legt fest, wie viele Versuche maximal erfolgen sollen, bevor Rückmeldungen gegeben werden müssen. Dieser Parameter ist optional und hat den Standardwert `1`. | | ||
| | `fileNameProperty` | Dieser Parameter steuert den Teil für das Herunterladen und Verifizieren der Dateien. Er akzeptiert drei Attribute. `@urlProperty` definiert den Namen der Vorgangseigenschaft, die die URL der Datei enthält. `@hashProperty` definiert den Namen der Vorgangseigenschaft, die die Checksumme der Datei enthält. Das Attribut `@folder` ist optional und hat den Standardwert `master`. Es steuert, wo die heruntergeladenen Dateien abgespeichert werden sollen. | | ||
| | `response` | Dieser optionale Parameter kann verwendet werden, um mehrere Rückmeldungen nach dem Downloaden und Verifizieren der Dateien zu geben. Er akzeptiert vier Attribute und einen JSON-Text für REST-Requests mit JSON-Body. Mehr Details und Beispiele sind innerhalb der Kommentare der beispielhaften Konfigurationsdatei ersichtlich. | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| --- | ||
| title: Downloading and verifying files | ||
| identifier: intranda_step_download_and_verify_assets | ||
| published: true | ||
| description: This step plugin makes it possible to download multiple files from URLs and verify them with checksums that have been defined as process properties. Feedback on the progress can be saved to different systems or locally within the journal. | ||
| --- | ||
| ## Introduction | ||
| This plugin reads URLs or hash values from several configured process properties, downloads the files from the defined URL and then compares them with the corresponding hash value. Finally, several responses can be given, depending on whether the status is `success` or `error`. These responses can be sent to another system via REST or simply logged within the journal. | ||
|
|
||
|
|
||
| ## Installation | ||
| To install the plugin, the following file must be installed: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/plugins/step/plugin_intranda_step_download_and_verify_assets-base.jar | ||
| ``` | ||
|
|
||
| The configuration file is usually located here: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/config/plugin_intranda_step_download_and_verify_assets.xml | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ## Configuration | ||
| The content of this configuration file looks like the following example: | ||
|
|
||
| ```xml | ||
| <config_plugin> | ||
| <!-- | ||
| order of configuration is: | ||
| 1.) project name and step name matches | ||
| 2.) step name matches and project is * | ||
| 3.) project name matches and step name is * | ||
| 4.) project name and step name are * | ||
| --> | ||
|
|
||
| <config> | ||
| <!-- which projects to use for (can be more then one, otherwise use *) --> | ||
| <project>*</project> | ||
| <step>*</step> | ||
|
|
||
| <!-- Configure here how many times shall be maximally tried before reporting final results. OPTIONAL. DEFAULT 1. --> | ||
| <maxTryTimes>3</maxTryTimes> | ||
|
|
||
| <!-- This tag accepts the following three attributes: | ||
| - @urlProperty: name of the property that holds the URL of the file | ||
| - @hashProperty: name of the property that holds the checksum of the file | ||
| - @folder: configured name of the target folder that shall be used to download the file. OPTIONAL. DEFAULT master. | ||
| --> | ||
| <fileNameProperty urlProperty="DraftUri" hashProperty="DraftHash" folder="master" /> | ||
| <fileNameProperty urlProperty="AssetUriSplitted" hashProperty="AssetHashSplitted" folder="master" /> | ||
|
|
||
| <!-- A response tag accepts four attributes: | ||
| - @type: success | error. Determines by which cases this configured response shall be activated. | ||
| - @method: OPTIONAL. If not configured or configured blankly, then the response will be performed via journal logs. Non-blank configuration options are: put | post | patch. | ||
| - @url: URL to the target system expecting this response. MANDATORY if @method is not blank.. | ||
| - @message: Message that shall be logged into journal. ONLY needed when @method is blank. | ||
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | ||
| One can also define a JSON string inside a pair of these tags, which will be used as JSON body to shoot a REST request. | ||
| --> | ||
| <!-- Usage of Goobi variables in @url as well as @message is allowed. --> | ||
| <response type="success" method="put" url="URL_ZU_BACH/upload_successful/{meta.ThesisId}" /> | ||
|
|
||
| <!-- For error cases there is no need for a response back to BACH, but an error message should be logged into journal. --> | ||
| <!-- Log ERROR_MESSAGE into journal as a signal of errors --> | ||
| <response type="error" message="ERROR_MESSAGE" /> | ||
|
|
||
| <!-- Example for REST calls with json body --> | ||
| <!-- | ||
| <response type="success" method="put" url="CHANGE_ME"> | ||
| { | ||
| "id": 0, | ||
| "name": "string", | ||
| "value": "string" | ||
| } | ||
| </response> | ||
| --> | ||
|
|
||
| </config> | ||
|
|
||
| </config_plugin> | ||
| ``` | ||
|
|
||
| The `<config>` block can occur repeatedly for different projects or work steps in order to be able to perform different actions within different workflows. | ||
|
|
||
| | Value | Description | | ||
| | :--- | :--- | | ||
| | `project` | This parameter defines which project the current block `<config>` should apply to. The name of the project is used here. This parameter can occur several times per `<config>` block. | | ||
| | `step` | This parameter controls which work steps the `<config>` block should apply to. The name of the work step is used here. This parameter can occur several times per `<config>` block. | | ||
| | `maxTryTimes` | This value defines the maximum number of attempts to be made before feedback must be given. This parameter is optional and has the default value `1`. | | ||
| | `fileNameProperty` | This parameter controls the part for downloading and verifying the files. It accepts three attributes. `@urlProperty` defines the name of the process property that contains the URL of the file. `@hashProperty` defines the name of the process property that contains the checksum of the file. The attribute `@folder` is optional and has the default value `master`. It controls where the downloaded files are to be saved. | | ||
| | `response` | This optional parameter can be used to provide multiple responses after downloading and verifying the files. It accepts four attributes and a JSON text for REST requests with a JSON body. More details and examples can be found in the comments of the sample configuration file. | |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters