-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
a9c0687
commit cc5b08f
Showing
2 changed files
with
186 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| --- | ||
| title: Download und Verifizieren von Dateien | ||
| identifier: intranda_step_download_and_verify_assets | ||
| published: true | ||
| description: Dieses Step-Plugin ermöglicht es, mehrere Dateien von URLs herunterzuladen und mit Checksummen zu verifizieren, die als Vorgangseigenschaften festgelegt wurden. Rückmeldungen über den Verlauf können zu verschiedenen Systemen oder auch lokal innerhalb des Journals gespeichert werden. | ||
| --- | ||
| ## Einführung | ||
| Dieses Plugin liest URLs bzw. Hash-Werte aus mehreren konfigurierten Vorgangseigenschaften ein, lädt die Dateien von der definierten URL herunter und verglicht sie anschließend mit dem zugehörigen Hash-Wert. Abschließend können mehrere Rückmeldungen gegeben werden, je nachdem ob der Status `success` oder `error` lautet. Diese Rückmeldungen können per REST zu einem anderen System geschickt oder einfach innerhalb des Journals geloggt werden. | ||
|
|
||
|
|
||
| ## Installation | ||
| Zur Installation des Plugins muss die folgende Datei installiert werden: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/plugins/step/plugin_intranda_step_download_and_verify_assets-base.jar | ||
| ``` | ||
|
|
||
| Die Konfigurationsdatei befindet sich üblicherweise hier: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/config/plugin_intranda_step_download_and_verify_assets.xml | ||
| ``` | ||
|
|
||
|
|
||
| ## Konfiguration | ||
| Der Inhalt dieser Konfigurationsdatei sieht beispielhaft wie folgt aus: | ||
|
|
||
| ```xml | ||
| <config_plugin> | ||
| <!-- | ||
| order of configuration is: | ||
| 1.) project name and step name matches | ||
| 2.) step name matches and project is * | ||
| 3.) project name matches and step name is * | ||
| 4.) project name and step name are * | ||
| --> | ||
|
|
||
| <config> | ||
| <!-- which projects to use for (can be more then one, otherwise use *) --> | ||
| <project>*</project> | ||
| <step>*</step> | ||
|
|
||
| <!-- Configure here how many times shall be maximally tried before reporting final results. OPTIONAL. DEFAULT 1. --> | ||
| <maxTryTimes>3</maxTryTimes> | ||
|
|
||
| <!-- This tag accepts the following three attributes: | ||
| - @urlProperty: name of the property that holds the URL of the file | ||
| - @hashProperty: name of the property that holds the checksum of the file | ||
| - @folder: configured name of the target folder that shall be used to download the file. OPTIONAL. DEFAULT master. | ||
| --> | ||
| <fileNameProperty urlProperty="DraftUri" hashProperty="DraftHash" folder="master" /> | ||
| <fileNameProperty urlProperty="AssetUriSplitted" hashProperty="AssetHashSplitted" folder="master" /> | ||
|
|
||
| <!-- A response tag accepts four attributes: | ||
| - @type: success | error. Determines by which cases this configured response shall be activated. | ||
| - @method: OPTIONAL. If not configured or configured blankly, then the response will be performed via journal logs. Non-blank configuration options are: put | post | patch. | ||
| - @url: URL to the target system expecting this response. MANDATORY if @method is not blank.. | ||
| - @message: Message that shall be logged into journal. ONLY needed when @method is blank. | ||
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | ||
| One can also define a JSON string inside a pair of these tags, which will be used as JSON body to shoot a REST request. | ||
| --> | ||
| <!-- Usage of Goobi variables in @url as well as @message is allowed. --> | ||
| <response type="success" method="put" url="URL_ZU_BACH/upload_successful/{meta.ThesisId}" /> | ||
|
|
||
| <!-- For error cases there is no need for a response back to BACH, but an error message should be logged into journal. --> | ||
| <!-- Log ERROR_MESSAGE into journal as a signal of errors --> | ||
| <response type="error" message="ERROR_MESSAGE" /> | ||
|
|
||
| <!-- Example for REST calls with json body --> | ||
| <!-- | ||
| <response type="success" method="put" url="CHANGE_ME"> | ||
| { | ||
| "id": 0, | ||
| "name": "string", | ||
| "value": "string" | ||
| } | ||
| </response> | ||
| --> | ||
|
|
||
| </config> | ||
|
|
||
| </config_plugin> | ||
| ``` | ||
|
|
||
| Der Block `<config>` kann für verschiedene Projekte oder Arbeitsschritte wiederholt vorkommen, um innerhalb verschiedener Workflows unterschiedliche Aktionen durchführen zu können. | ||
|
|
||
| | Wert | Beschreibung | | ||
| | :--- | :--- | | ||
| | `project` | Dieser Parameter legt fest, für welches Projekt der aktuelle Block `<config>` gelten soll. Verwendet wird hierbei der Name des Projektes. Dieser Parameter kann mehrfach pro `<config>` Block vorkommen. | | ||
| | `step` | Dieser Parameter steuert, für welche Arbeitsschritte der Block `<config>` gelten soll. Verwendet wird hier der Name des Arbeitsschritts. Dieser Parameter kann mehrfach pro `<config>` Block vorkommen. | | ||
| | `maxTryTimes` | Dieser Wert legt fest, wie viele Versuche maximal erfolgen sollen, bevor Rückmeldungen gegeben werden müssen. Dieser Parameter ist optional und hat den Standardwert `1`. | | ||
| | `fileNameProperty` | Dieser Parameter steuert den Teil für das Herunterladen und Verifizieren der Dateien. Er akzeptiert drei Attribute. `@urlProperty` definiert den Namen der Vorgangseigenschaft, die die URL der Datei enthält. `@hashProperty` definiert den Namen der Vorgangseigenschaft, die die Checksumme der Datei enthält. Das Attribut `@folder` ist optional und hat den Standardwert `master`. Es steuert, wo die heruntergeladenen Dateien abgespeichert werden sollen. | | ||
| | `response` | Dieser optionale Parameter kann verwendet werden, um mehrere Rückmeldungen nach dem Downloaden und Verifizieren der Dateien zu geben. Er akzeptiert vier Attribute und einen JSON-Text für REST-Requests mit JSON-Body. Mehr Details und Beispiele sind innerhalb der Kommentare der beispielhaften Konfigurationsdatei ersichtlich. | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| --- | ||
| title: Downloading and verifying files | ||
| identifier: intranda_step_download_and_verify_assets | ||
| published: true | ||
| description: This step plugin makes it possible to download multiple files from URLs and verify them with checksums that have been defined as process properties. Feedback on the progress can be saved to different systems or locally within the journal. | ||
| --- | ||
| ## Introduction | ||
| This plugin reads URLs or hash values from several configured process properties, downloads the files from the defined URL and then compares them with the corresponding hash value. Finally, several responses can be given, depending on whether the status is `success` or `error`. These responses can be sent to another system via REST or simply logged within the journal. | ||
|
|
||
|
|
||
| ## Installation | ||
| To install the plugin, the following file must be installed: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/plugins/step/plugin_intranda_step_download_and_verify_assets-base.jar | ||
| ``` | ||
|
|
||
| The configuration file is usually located here: | ||
|
|
||
| ```bash | ||
| /opt/digiverso/goobi/config/plugin_intranda_step_download_and_verify_assets.xml | ||
| ``` | ||
|
|
||
|
|
||
| ## Configuration | ||
| The content of this configuration file looks like the following example: | ||
|
|
||
| ```xml | ||
| <config_plugin> | ||
| <!-- | ||
| order of configuration is: | ||
| 1.) project name and step name matches | ||
| 2.) step name matches and project is * | ||
| 3.) project name matches and step name is * | ||
| 4.) project name and step name are * | ||
| --> | ||
|
|
||
| <config> | ||
| <!-- which projects to use for (can be more then one, otherwise use *) --> | ||
| <project>*</project> | ||
| <step>*</step> | ||
|
|
||
| <!-- Configure here how many times shall be maximally tried before reporting final results. OPTIONAL. DEFAULT 1. --> | ||
| <maxTryTimes>3</maxTryTimes> | ||
|
|
||
| <!-- This tag accepts the following three attributes: | ||
| - @urlProperty: name of the property that holds the URL of the file | ||
| - @hashProperty: name of the property that holds the checksum of the file | ||
| - @folder: configured name of the target folder that shall be used to download the file. OPTIONAL. DEFAULT master. | ||
| --> | ||
| <fileNameProperty urlProperty="DraftUri" hashProperty="DraftHash" folder="master" /> | ||
| <fileNameProperty urlProperty="AssetUriSplitted" hashProperty="AssetHashSplitted" folder="master" /> | ||
|
|
||
| <!-- A response tag accepts four attributes: | ||
| - @type: success | error. Determines by which cases this configured response shall be activated. | ||
| - @method: OPTIONAL. If not configured or configured blankly, then the response will be performed via journal logs. Non-blank configuration options are: put | post | patch. | ||
| - @url: URL to the target system expecting this response. MANDATORY if @method is not blank.. | ||
| - @message: Message that shall be logged into journal. ONLY needed when @method is blank. | ||
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | ||
| One can also define a JSON string inside a pair of these tags, which will be used as JSON body to shoot a REST request. | ||
| --> | ||
| <!-- Usage of Goobi variables in @url as well as @message is allowed. --> | ||
| <response type="success" method="put" url="URL_ZU_BACH/upload_successful/{meta.ThesisId}" /> | ||
|
|
||
| <!-- For error cases there is no need for a response back to BACH, but an error message should be logged into journal. --> | ||
| <!-- Log ERROR_MESSAGE into journal as a signal of errors --> | ||
| <response type="error" message="ERROR_MESSAGE" /> | ||
|
|
||
| <!-- Example for REST calls with json body --> | ||
| <!-- | ||
| <response type="success" method="put" url="CHANGE_ME"> | ||
| { | ||
| "id": 0, | ||
| "name": "string", | ||
| "value": "string" | ||
| } | ||
| </response> | ||
| --> | ||
|
|
||
| </config> | ||
|
|
||
| </config_plugin> | ||
| ``` | ||
|
|
||
| The `<config>` block can occur repeatedly for different projects or work steps in order to be able to perform different actions within different workflows. | ||
|
|
||
| | Value | Description | | ||
| | :--- | :--- | | ||
| | `project` | This parameter defines which project the current block `<config>` should apply to. The name of the project is used here. This parameter can occur several times per `<config>` block. | | ||
| | `step` | This parameter controls which work steps the `<config>` block should apply to. The name of the work step is used here. This parameter can occur several times per `<config>` block. | | ||
| | `maxTryTimes` | This value defines the maximum number of attempts to be made before feedback must be given. This parameter is optional and has the default value `1`. | | ||
| | `fileNameProperty` | This parameter controls the part for downloading and verifying the files. It accepts three attributes. `@urlProperty` defines the name of the process property that contains the URL of the file. `@hashProperty` defines the name of the process property that contains the checksum of the file. The attribute `@folder` is optional and has the default value `master`. It controls where the downloaded files are to be saved. | | ||
| | `response` | This optional parameter can be used to provide multiple responses after downloading and verifying the files. It accepts four attributes and a JSON text for REST requests with a JSON body. More details and examples can be found in the comments of the sample configuration file. | |