Maint 4.x fixup! feat download wfs files to tmp folder for bucket service #1049

.github/workflows/check.yml

@@ -0,0 +1,43 @@
+name: Check
+
+# XXX disabled for now because we use Jenkins still, but at the time this was tested it was functional
+# on:
+#   pull_request:
+#     branches: [ master ]
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+
+    # Note: Actually we don't need Maven CLI, only Java 17, but this may have additional benefits due to the Maven cache
+    - name: Setup Maven
+      uses: s4u/[email protected]
+      with:
+        java-version: 17
+        java-distribution: temurin
+        maven-version: 3.8.6
+
+    - name: Clean
+      run: ./build.sh clean
+      working-directory: ./build
+
+    - name: Test (commitStage)
+      run: ./build.sh commitStage
+      # run: ./build.sh integrationStage
+      working-directory: ./build
+
+      # https://github.com/marketplace/actions/junit-report-action
+    - name: Publish Test Report
+      uses: mikepenz/action-junit-report@v3
+      if: always() # always run even if the previous step fails
+      with:
+        report_paths: 'build/target/testReports/*.xml'
+
+      # TODO allure report?
+      # build/target/allure-results
+
+      # TODO archive logs?
+      # build/target/testReports/*.out,build/target/testReports/*.err

.gitignore

@@ -38,3 +38,5 @@ Thumbs.db
 *.swp
 
 #.gitignore
+*.hprof
+*.tycho-consumer-pom.xml

CHANGELOG.md

@@ -4,6 +4,58 @@ See the [change log guidelines](http://keepachangelog.com/) for information on h
 
 ## [Unreleased]
 
+## [5.0.1]
+
+### Added
+- Add an option to overwrite content in Geopackage Writer instead of appending content
+- Add a parameter to the GeoPackage writer that allows creating tables for all mapping-relevant target types
+- Create a code page (.cpg) file when exporting a Shapefile
+
+### Fixed
+- Fix the Commons Text security vulnerability
+- Fix to Proxy settings to be able to contact update sites when Proxy usage is required
+- Fix the automation of the build process
+
+
+## [5.0.0]
+
+### Added
+- Support for Java 17
+- Support for Groovy 2.5
+- Support to configure details on HTTP client max connections and connection eviction via environment variables
+- Support collecting metrics on Request class HTTP clients 
+- Support for TopoJSON as a format for exporting data
+- Support to get list of Shapefile names
+- Support for XPlanGML 5.4 and XPlanGML 6.0 schema preset
+- Support to allow excluding root properties from structural Retype
+- Support to automatically add the codespace for INSPIRE feature types
+- Support for `xs:all` to the XML Schema Reader
+- Support to specify initial value during "Generate sequential ID" function
+- Support to read CRS information from Geopackage file rather than reading it from the schema
+- Enhanced support for loading XML/GML schema resembling Geopackage structure so that no information is lost
+- Support to specify leniency when using DateExtraction function 
+- Support to skip n-lines when reading csv and xls files
+
+### Changed
+- Merge schemas for HSD export as XML or Json to one schema to work around the restriction that only single schema can be imported from HSD file
+
+### Removed
+- Deegree workspace configuration generation has been removed since it requires further work to integrate updated dependencies that are compatible with Java 17
+
+### Fixed
+- Fix to check for a null date when converting a string to any form of date and/or time
+- Fix to add exception message to the transformation log 
+- Fix to propose one single variable or a list during Groovy merge
+- Fix to remove group property definitions from path when using Add condition on context function
+- Fix to not truncate Shapefile attributes if their length is less than 10 characters
+- Fix to correct the winding order as counterclockwise based
+on the right-hand rule when exporting data in Json format
+- Fix to preserve original order of columns during CSV export
+- Fix to clean progress bar when opening a new project after exporting in CSV and XLS
+- Fix to handle NPE when exporting multiple geometries to Shapefiles
+- Fix to apply queryFilter when reading Geopackage files instead of writing them
+- Fix to allow loading Geopackage files with non standard file extension
+
 ## [4.1.0]
 
 ### Added
@@ -397,7 +449,8 @@ See the [change log guidelines](http://keepachangelog.com/) for information on h
 
 Changes so far have been documented in the [hale help](http://hale.igd.fraunhofer.de/2.9.4/help/topic/eu.esdihumboldt.hale.doc.user/html/new/2_9_0.xhtml?cp=2_1_0).
 
-[Unreleased]: https://github.com/halestudio/hale/compare/4.1.0...HEAD
+[Unreleased]: https://github.com/halestudio/hale/compare/5.0.0...HEAD
+[5.0.0]: https://github.com/halestudio/hale/compare/4.1.0...5.0.0
 [4.1.0]: https://github.com/halestudio/hale/compare/4.0.0...4.1.0
 [4.0.0]: https://github.com/halestudio/hale/compare/3.5.0...4.0.0
 [3.5.0]: https://github.com/halestudio/hale/compare/3.4.1...3.5.0

README.md

@@ -37,8 +37,8 @@ To contribute please check the [contribution guidelines](CONTRIBUTING.md) and re
 
 ## System Requirements
 
-* Windows XP/Vista/7/8 32bit or 64bit
-* MacOS X 10.5
+* Windows XP/Vista/7/8/10/11 32bit or 64bit
+* MacOS X 10.5+
 * Linux 
 * 4 GB RAM recommended
 

RELEASE.md

@@ -9,9 +9,9 @@ Before proceeding with the release steps
 
 Create a new branch from the master:
 
-1. Create a PR to update the [CHANGELOG.md](https://github.com/kapil-agnihotri/hale/blob/master/CHANGELOG.md)
+1. Create a PR to update the [CHANGELOG.md](https://github.com/halestudio/hale/blob/master/CHANGELOG.md#change-log)
 2. Create a PR to update [what's new page](https://github.com/halestudio/hale/tree/master/doc/plugins/eu.esdihumboldt.hale.doc.user/html/new) by creating a new `<release-tag>.xhtml` page.
-3. Once the PRs are merged to the master, update the Infocenter container, so that [what's new page in the documentation](http://help.halestudio.org/latest/index.jsp?topic=%2Feu.esdihumboldt.hale.doc.user%2Fhtml%2Fnew%2F4_1_0.xhtml&cp%3D0_1_0) is updated to show the latest release information. Then proceed with the steps for a HALE release.
+3. Once the PRs are merged to the master, update the [Infocenter container](https://github.com/halestudio/hale/blob/fb07374fdd4e5078ccbc2074fd8a13bd48982e58/doc/plugins/eu.esdihumboldt.hale.doc.user/toc.xml#L17), so that [what's new page in the documentation](http://help.halestudio.org/latest/index.jsp?topic=%2Feu.esdihumboldt.hale.doc.user%2Fhtml%2Fnew%2F4_1_0.xhtml&cp%3D0_1_0) is updated to show the latest release information. Then proceed with the steps for a HALE release.
 
 Steps for a HALE release
 ========================
@@ -56,6 +56,8 @@ On master branch:
 
 After the release
 =================
-1. Create Windows installer
+1. Create Windows installer.
+Prerequisites: install Wix Toolset v.3.11 [available here](https://github.com/wixtoolset/wix3/releases/tag/wix3112rtm)
+Then run `build.bat product -o windows -a x86_64 HALE` from the root directory.
 2. Draft a new release for the latest release at https://github.com/halestudio/hale/releases and publish it.
 3. Update [download page](https://github.com/wetransform/www.wetransform.to/blob/deploy/app/downloads/index.html) so that the latest builds are available for download from https://www.wetransform.to/downloads/

app/plugins/eu.esdihumboldt.hale.app.cli.commands/.classpath

@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <classpath>
-	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8"/>
+	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-17"/>
 	<classpathentry kind="con" path="org.eclipse.pde.core.requiredPlugins"/>
 	<classpathentry kind="src" path="src"/>
 	<classpathentry exported="true" kind="con" path="GROOVY_DSL_SUPPORT"/>

app/plugins/eu.esdihumboldt.hale.app.cli.commands/.settings/org.eclipse.jdt.core.prefs

@@ -1,5 +1,5 @@
-#Updated from default preferences Apr 25, 2017 3:25:14 PM
-#Tue Apr 25 15:25:14 CEST 2017
+#Updated from default preferences 28 Oct 2022, 08:10:30
+#Fri Oct 28 08:10:30 CEST 2022
 eclipse.preferences.version=1
 org.eclipse.jdt.core.builder.cleanOutputFolder=clean
 org.eclipse.jdt.core.builder.duplicateResourceTask=warning
@@ -21,9 +21,9 @@ org.eclipse.jdt.core.codeComplete.staticFinalFieldPrefixes=
 org.eclipse.jdt.core.codeComplete.staticFinalFieldSuffixes=
 org.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode=enabled
 org.eclipse.jdt.core.compiler.codegen.methodParameters=do not generate
-org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.8
+org.eclipse.jdt.core.compiler.codegen.targetPlatform=17
 org.eclipse.jdt.core.compiler.codegen.unusedLocal=preserve
-org.eclipse.jdt.core.compiler.compliance=1.8
+org.eclipse.jdt.core.compiler.compliance=17
 org.eclipse.jdt.core.compiler.debug.lineNumber=generate
 org.eclipse.jdt.core.compiler.debug.localVariable=generate
 org.eclipse.jdt.core.compiler.debug.sourceFile=generate
@@ -39,6 +39,7 @@ org.eclipse.jdt.core.compiler.problem.deprecationInDeprecatedCode=disabled
 org.eclipse.jdt.core.compiler.problem.deprecationWhenOverridingDeprecatedMethod=enabled
 org.eclipse.jdt.core.compiler.problem.discouragedReference=warning
 org.eclipse.jdt.core.compiler.problem.emptyStatement=warning
+org.eclipse.jdt.core.compiler.problem.enablePreviewFeatures=disabled
 org.eclipse.jdt.core.compiler.problem.enumIdentifier=error
 org.eclipse.jdt.core.compiler.problem.fallthroughCase=ignore
 org.eclipse.jdt.core.compiler.problem.fatalOptionalError=enabled
@@ -81,6 +82,7 @@ org.eclipse.jdt.core.compiler.problem.potentialNullReference=warning
 org.eclipse.jdt.core.compiler.problem.rawTypeReference=warning
 org.eclipse.jdt.core.compiler.problem.redundantNullCheck=warning
 org.eclipse.jdt.core.compiler.problem.redundantSuperinterface=ignore
+org.eclipse.jdt.core.compiler.problem.reportPreviewFeatures=warning
 org.eclipse.jdt.core.compiler.problem.specialParameterHidingField=disabled
 org.eclipse.jdt.core.compiler.problem.staticAccessReceiver=warning
 org.eclipse.jdt.core.compiler.problem.suppressOptionalErrors=disabled
@@ -108,8 +110,10 @@ org.eclipse.jdt.core.compiler.problem.unusedParameterWhenOverridingConcrete=disa
 org.eclipse.jdt.core.compiler.problem.unusedPrivateMember=warning
 org.eclipse.jdt.core.compiler.problem.unusedWarningToken=warning
 org.eclipse.jdt.core.compiler.problem.varargsArgumentNeedCast=warning
-org.eclipse.jdt.core.compiler.source=1.8
+org.eclipse.jdt.core.compiler.release=enabled
+org.eclipse.jdt.core.compiler.source=17
 org.eclipse.jdt.core.formatter.align_type_members_on_columns=false
+org.eclipse.jdt.core.formatter.alignment_for_additive_operator=16
 org.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression=16
 org.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation=0
 org.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant=16
@@ -118,16 +122,20 @@ org.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation=16
 org.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression=16
 org.eclipse.jdt.core.formatter.alignment_for_assignment=0
 org.eclipse.jdt.core.formatter.alignment_for_binary_expression=16
+org.eclipse.jdt.core.formatter.alignment_for_bitwise_operator=16
 org.eclipse.jdt.core.formatter.alignment_for_compact_if=16
 org.eclipse.jdt.core.formatter.alignment_for_conditional_expression=80
 org.eclipse.jdt.core.formatter.alignment_for_enum_constants=0
 org.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer=16
+org.eclipse.jdt.core.formatter.alignment_for_logical_operator=16
 org.eclipse.jdt.core.formatter.alignment_for_method_declaration=0
 org.eclipse.jdt.core.formatter.alignment_for_multiple_fields=16
+org.eclipse.jdt.core.formatter.alignment_for_multiplicative_operator=16
 org.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration=16
 org.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration=16
 org.eclipse.jdt.core.formatter.alignment_for_resources_in_try=80
 org.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation=16
+org.eclipse.jdt.core.formatter.alignment_for_string_concatenation=16
 org.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration=16
 org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration=16
 org.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration=16
@@ -213,11 +221,13 @@ org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_constant=insert
 org.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_declaration=insert
 org.eclipse.jdt.core.formatter.insert_new_line_in_empty_method_body=insert
 org.eclipse.jdt.core.formatter.insert_new_line_in_empty_type_declaration=insert
+org.eclipse.jdt.core.formatter.insert_space_after_additive_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter=insert
 org.eclipse.jdt.core.formatter.insert_space_after_assignment_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_binary_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_after_bitwise_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments=insert
 org.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters=insert
 org.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block=insert
@@ -248,6 +258,8 @@ org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments=insert
 org.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters=insert
 org.eclipse.jdt.core.formatter.insert_space_after_ellipsis=insert
 org.eclipse.jdt.core.formatter.insert_space_after_lambda_arrow=insert
+org.eclipse.jdt.core.formatter.insert_space_after_logical_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_after_multiplicative_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters=do not insert
@@ -272,13 +284,18 @@ org.eclipse.jdt.core.formatter.insert_space_after_postfix_operator=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_prefix_operator=do not insert
 org.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional=insert
 org.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard=do not insert
+org.eclipse.jdt.core.formatter.insert_space_after_relational_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for=insert
 org.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_try_resources=insert
+org.eclipse.jdt.core.formatter.insert_space_after_shift_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_after_string_concatenation=insert
 org.eclipse.jdt.core.formatter.insert_space_after_unary_operator=do not insert
+org.eclipse.jdt.core.formatter.insert_space_before_additive_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter=insert
 org.eclipse.jdt.core.formatter.insert_space_before_assignment_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration=insert
 org.eclipse.jdt.core.formatter.insert_space_before_binary_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_before_bitwise_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters=do not insert
@@ -326,6 +343,8 @@ org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments=do no
 org.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_ellipsis=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_lambda_arrow=insert
+org.eclipse.jdt.core.formatter.insert_space_before_logical_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_before_multiplicative_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters=do not insert
@@ -362,9 +381,12 @@ org.eclipse.jdt.core.formatter.insert_space_before_postfix_operator=do not inser
 org.eclipse.jdt.core.formatter.insert_space_before_prefix_operator=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional=insert
 org.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard=do not insert
+org.eclipse.jdt.core.formatter.insert_space_before_relational_operator=insert
 org.eclipse.jdt.core.formatter.insert_space_before_semicolon=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for=do not insert
 org.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_try_resources=do not insert
+org.eclipse.jdt.core.formatter.insert_space_before_shift_operator=insert
+org.eclipse.jdt.core.formatter.insert_space_before_string_concatenation=insert
 org.eclipse.jdt.core.formatter.insert_space_before_unary_operator=do not insert
 org.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference=do not insert
 org.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer=do not insert
@@ -390,8 +412,13 @@ org.eclipse.jdt.core.formatter.tabulation.char=tab
 org.eclipse.jdt.core.formatter.tabulation.size=4
 org.eclipse.jdt.core.formatter.use_on_off_tags=false
 org.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations=false
+org.eclipse.jdt.core.formatter.wrap_before_additive_operator=true
 org.eclipse.jdt.core.formatter.wrap_before_binary_operator=true
+org.eclipse.jdt.core.formatter.wrap_before_bitwise_operator=true
+org.eclipse.jdt.core.formatter.wrap_before_logical_operator=true
+org.eclipse.jdt.core.formatter.wrap_before_multiplicative_operator=true
 org.eclipse.jdt.core.formatter.wrap_before_or_operator_multicatch=true
+org.eclipse.jdt.core.formatter.wrap_before_string_concatenation=true
 org.eclipse.jdt.core.formatter.wrap_outer_expressions_when_nested=true
 org.eclipse.jdt.core.incompatibleJDKLevel=ignore
 org.eclipse.jdt.core.incompleteClasspath=error

app/plugins/eu.esdihumboldt.hale.app.cli.commands/.settings/org.eclipse.jdt.groovy.core.prefs

@@ -0,0 +1,4 @@
+#Created from default preferences 28 Oct 2022, 08:10:30
+#Fri Oct 28 08:10:30 CEST 2022
+eclipse.preferences.version=1
+groovy.compiler.level=25

app/plugins/eu.esdihumboldt.hale.app.cli.commands/META-INF/MANIFEST.MF

@@ -2,12 +2,12 @@ Manifest-Version: 1.0
 Bundle-ManifestVersion: 2
 Bundle-Name: hale CLI Default Commands
 Bundle-SymbolicName: eu.esdihumboldt.hale.app.cli.commands;singleton:=true
-Bundle-Version: 4.2.0.qualifier
+Bundle-Version: 5.1.0.qualifier
 Bundle-Vendor: wetransform GmbH
 Bundle-RequiredExecutionEnvironment: JavaSE-1.8
 Require-Bundle: eu.esdihumboldt.util.cli;bundle-version="2.9.5",
  eu.esdihumboldt.hale.common.cli;bundle-version="2.9.5",
- groovy;bundle-version="2.3.7",
+ groovy;bundle-version="2.5.19",
  eu.esdihumboldt.hale.common.align;bundle-version="2.9.5",
  eu.esdihumboldt.hale.io.html.svg;bundle-version="2.9.5",
  eu.esdihumboldt.hale.io.xls;bundle-version="2.9.5",

app/plugins/eu.esdihumboldt.hale.app.cli.commands/src/eu/esdihumboldt/hale/app/cli/commands/alignment/ExportJsonCommand.groovy

@@ -22,6 +22,7 @@ import eu.esdihumboldt.hale.common.core.report.ReportHandler
 import eu.esdihumboldt.hale.common.headless.impl.ProjectTransformationEnvironment
 import eu.esdihumboldt.hale.io.html.svg.mapping.json.JsonMappingExporter
 import eu.esdihumboldt.util.cli.CommandContext
+import groovy.cli.picocli.OptionAccessor
 import groovy.transform.CompileStatic
 
 /**