Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restore classic Join and Merge handlers, make index-based handlers optional #1275

Merged
merged 2 commits into from
Nov 11, 2024
Merged

Restore classic Join and Merge handlers, make index-based handlers optional #1275

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
aee64ff
perf: make index join/merge handlers optional
florianesser Nov 7, 2024
a688c7c
docs: add documentation for index join/merge handlers
florianesser Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions common/plugins/eu.esdihumboldt.hale.common.align/help/join.xhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,19 @@
</div>
<p>To create a condition, select a property on the left and on the
right side, then click on the button in the middle.</p>

<h2>
<a name="condition">Index Join Handler (experimental)</a>
</h2>
<p>
To activate the use of the Index Join Handler for all Join and Groovy
Join transformations, set the Java system property <code>hale.functions.use_index_join_handler</code>
or the environment variable <code>HALE_FUNCTIONS_USE_INDEX_JOIN_HANDLER</code>.
</p>
<p>
The Index Join Handler is experimental and can potentially have a
negative impact on the performance of Join and Groovy Join transformations.
</p>
</div>
</body>
</html>
14 changes: 14 additions & 0 deletions common/plugins/eu.esdihumboldt.hale.common.align/help/merge/help.xhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,20 @@
</br>
<img src="../../eu.esdihumboldt.hale.common.align/help/merge/mergeAggregate.png" />
</p>

<h2>
<a name="condition">Index Merge Handler (experimental)</a>
</h2>
<p>
To activate the use of the Index Merge Handler for all Merge and Groovy
Merge transformations, set the Java system property <code>hale.functions.use_index_merge_handler</code>
or the environment variable <code>HALE_FUNCTIONS_USE_INDEX_MERGE_HANDLER</code>.
</p>
<p>
The Index Merge Handler is experimental and can potentially have a
negative impact on the performance of Merge and Groovy Merge transformations.
</p>

</div>
</body>
</html>
2 changes: 2 additions & 0 deletions cst/plugins/eu.esdihumboldt.cst.functions.core/META-INF/MANIFEST.MF
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Require-Bundle: eu.esdihumboldt.hale.common.align;bundle-version="2.2.0",
Import-Package: com.google.common.base;version="9.0.0",
com.google.common.collect,
de.fhg.igd.osgi.util;version="1.0.0",
de.fhg.igd.slf4jplus,
eu.esdihumboldt.hale.common.convert,
eu.esdihumboldt.hale.common.core,
eu.esdihumboldt.hale.common.core.io,
Expand All @@ -34,6 +35,7 @@ Import-Package: com.google.common.base;version="9.0.0",
eu.esdihumboldt.util.groovy.paths,
net.jcip.annotations,
org.locationtech.jts.geom;version="1.13.0",
org.slf4j;version="1.7.36",
org.springframework.core.convert;version="5.2.0"
Export-Package: eu.esdihumboldt.cst.functions.core,
eu.esdihumboldt.cst.functions.core.join,
Expand Down
28 changes: 26 additions & 2 deletions ...ugins/eu.esdihumboldt.cst.functions.core/src/eu/esdihumboldt/cst/functions/core/Join.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@
import java.util.Collections;
import java.util.List;

import de.fhg.igd.slf4jplus.ALogger;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build error hints at package import may be missing in manifest?

import de.fhg.igd.slf4jplus.ALoggerFactory;
import eu.esdihumboldt.cst.functions.core.join.IndexJoinHandler;
import eu.esdihumboldt.cst.functions.core.join.JoinHandler;
import eu.esdihumboldt.hale.common.align.model.Cell;
import eu.esdihumboldt.hale.common.align.model.functions.JoinFunction;
import eu.esdihumboldt.hale.common.align.model.functions.join.JoinParameter;
Expand All @@ -34,17 +37,38 @@
/**
* Type transformation that joins multiple instances of different source types
* into one target instance, based on matching properties.
*
*
* @author Kai Schwierczek
*/
public class Join extends Retype implements JoinFunction, InstanceIndexContribution {

/**
* The log
*/
private static final ALogger LOG = ALoggerFactory.getLogger(Join.class);

/**
* @see eu.esdihumboldt.hale.common.align.transformation.function.impl.AbstractTypeTransformation#getInstanceHandler()
*/
@Override
public InstanceHandler<? super TransformationEngine> getInstanceHandler() {
return new IndexJoinHandler();
boolean useIndexJoinHandler = false;

String setting = System.getProperty("hale.functions.join.use_index_join_handler");

if (setting == null) {
setting = System.getenv("HALE_FUNCTIONS_USE_INDEX_JOIN_HANDLER");
}

if (setting != null) {
try {
useIndexJoinHandler = Boolean.valueOf(setting);
} catch (Throwable e) {
LOG.error("Error applying index join handler setting: " + setting, e);
}
}

return useIndexJoinHandler ? new IndexJoinHandler() : new JoinHandler();
}

/**
Expand Down
28 changes: 26 additions & 2 deletions ...gins/eu.esdihumboldt.cst.functions.core/src/eu/esdihumboldt/cst/functions/core/Merge.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,10 @@
import java.util.Collections;
import java.util.List;

import de.fhg.igd.slf4jplus.ALogger;
import de.fhg.igd.slf4jplus.ALoggerFactory;
import eu.esdihumboldt.cst.functions.core.merge.IndexMergeHandler;
import eu.esdihumboldt.cst.functions.core.merge.PropertiesMergeHandler;
import eu.esdihumboldt.hale.common.align.model.Cell;
import eu.esdihumboldt.hale.common.align.model.functions.MergeFunction;
import eu.esdihumboldt.hale.common.align.model.functions.merge.MergeUtil;
Expand All @@ -32,17 +35,38 @@
/**
* Type transformation that merges multiple instances of the same source type
* into one target instance, based on matching properties.
*
*
* @author Simon Templer
*/
public class Merge extends Retype implements MergeFunction, InstanceIndexContribution {

/**
* The log
*/
private static final ALogger LOG = ALoggerFactory.getLogger(Merge.class);

/**
* @see eu.esdihumboldt.hale.common.align.transformation.function.impl.AbstractTypeTransformation#getInstanceHandler()
*/
@Override
public InstanceHandler<? super TransformationEngine> getInstanceHandler() {
return new IndexMergeHandler();
boolean useIndexMergeHandler = false;

String setting = System.getProperty("hale.functions.use_index_merge_handler");

if (setting == null) {
setting = System.getenv("HALE_FUNCTIONS_USE_INDEX_MERGE_HANDLER");
}

if (setting != null) {
try {
useIndexMergeHandler = Boolean.valueOf(setting);
} catch (Throwable e) {
LOG.error("Error applying index merge handler setting: " + setting, e);
}
}

return useIndexMergeHandler ? new IndexMergeHandler() : new PropertiesMergeHandler();
}

/**
Expand Down
28 changes: 26 additions & 2 deletions ...sdihumboldt.cst.functions.groovy/src/eu/esdihumboldt/cst/functions/groovy/GroovyJoin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@
import java.util.Collection;
import java.util.List;

import de.fhg.igd.slf4jplus.ALogger;
import de.fhg.igd.slf4jplus.ALoggerFactory;
import eu.esdihumboldt.cst.functions.core.Join;
import eu.esdihumboldt.cst.functions.core.join.IndexJoinHandler;
import eu.esdihumboldt.cst.functions.core.join.JoinHandler;
import eu.esdihumboldt.hale.common.align.model.Cell;
import eu.esdihumboldt.hale.common.align.model.functions.JoinFunction;
import eu.esdihumboldt.hale.common.align.model.impl.PropertyEntityDefinition;
Expand All @@ -32,11 +35,16 @@
* into one target instance, based on matching properties. The transformation
* also applies a Groovy script that can be used to control the target instance
* creation.
*
*
* @author Simon Templer
*/
public class GroovyJoin extends GroovyRetype implements JoinFunction, InstanceIndexContribution {

/**
* The log
*/
private static final ALogger LOG = ALoggerFactory.getLogger(GroovyJoin.class);

/**
* The function ID. Not named <code>ID</code> to avoid shadowing
* {@link JoinFunction#ID}.
Expand All @@ -50,7 +58,23 @@ public class GroovyJoin extends GroovyRetype implements JoinFunction, InstanceIn

@Override
public InstanceHandler<? super TransformationEngine> getInstanceHandler() {
return new IndexJoinHandler();
boolean useIndexJoinHandler = false;

String setting = System.getProperty("hale.functions.use_index_join_handler");

if (setting == null) {
setting = System.getenv("HALE_FUNCTIONS_USE_INDEX_JOIN_HANDLER");
}

if (setting != null) {
try {
useIndexJoinHandler = Boolean.valueOf(setting);
} catch (Throwable e) {
LOG.error("Error applying index join handler setting: " + setting, e);
}
}

return useIndexJoinHandler ? new IndexJoinHandler() : new JoinHandler();
}

/**
Expand Down
28 changes: 26 additions & 2 deletions ...dihumboldt.cst.functions.groovy/src/eu/esdihumboldt/cst/functions/groovy/GroovyMerge.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@
import java.util.Collection;
import java.util.List;

import de.fhg.igd.slf4jplus.ALogger;
import de.fhg.igd.slf4jplus.ALoggerFactory;
import eu.esdihumboldt.cst.functions.core.Merge;
import eu.esdihumboldt.cst.functions.core.merge.IndexMergeHandler;
import eu.esdihumboldt.cst.functions.core.merge.PropertiesMergeHandler;
import eu.esdihumboldt.hale.common.align.model.Cell;
import eu.esdihumboldt.hale.common.align.model.functions.MergeFunction;
import eu.esdihumboldt.hale.common.align.model.impl.PropertyEntityDefinition;
Expand All @@ -33,11 +36,16 @@
* into one target instance, based on matching properties. The transformation
* also applies a Groovy script that can be used to control the target instance
* creation.
*
*
* @author Simon Templer
*/
public class GroovyMerge extends GroovyRetype implements MergeFunction, InstanceIndexContribution {

/**
* The log
*/
private static final ALogger LOG = ALoggerFactory.getLogger(GroovyMerge.class);

/**
* The function ID. Not named <code>ID</code> to avoid shadowing
* {@link MergeFunction#ID}.
Expand All @@ -51,7 +59,23 @@ public class GroovyMerge extends GroovyRetype implements MergeFunction, Instance

@Override
public InstanceHandler<? super TransformationEngine> getInstanceHandler() {
return new IndexMergeHandler();
boolean useIndexMergeHandler = false;

String setting = System.getProperty("hale.functions.use_index_merge_handler");

if (setting == null) {
setting = System.getenv("HALE_FUNCTIONS_USE_INDEX_MERGE_HANDLER");
}

if (setting != null) {
try {
useIndexMergeHandler = Boolean.valueOf(setting);
} catch (Throwable e) {
LOG.error("Error applying index merge handler setting: " + setting, e);
}
}

return useIndexMergeHandler ? new IndexMergeHandler() : new PropertiesMergeHandler();
}

/**
Expand Down
Loading