[SEDONA-630] Improve ST_Union_Aggr performance

[zhangfengcdt]

Did you read the Contributor Guide?

• Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

• Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-XXX. The PR name follows the format [SEDONA-XXX] my subject.

What changes were proposed in this PR?

Switch to JTS OverlayNGRobust.union function to perform geometry union and add geometry cache capability.
https://locationtech.github.io/jts/javadoc/org/locationtech/jts/operation/overlayng/OverlayNGRobust.html

How was this patch tested?

All existing unit tests should pass.

Did this PR include necessary documentation updates?

• No, this PR does not affect any public API so no need to change the documentation.

[zhangfengcdt]

@jiayuasu I noticed that after switching from geo.buffer to OverlayNGRobust.union, the complex geometry representation returned from ST_Union_Aggr might change due to the reordering of polygon/polyline vertex. For example:

New: POLYGON ((1 0, 0 0, 0 1, 1 1, 2 1, 2 0, 1 0))

Old: POLYGON ((0 0, 0 1, 1 1, 2 1, 2 0, 1 0, 0 0))

They represent the same polygon, but the vertex order has changed.

[jiayuasu]

@jiayuasu I noticed that after switching from geo.buffer to OverlayNGRobust.union, the complex geometry representation returned from ST_Union_Aggr might change due to the reordering of polygon/polyline vertex. For example:

New: POLYGON ((1 0, 0 0, 0 1, 1 1, 2 1, 2 0, 1 0))

Old: POLYGON ((0 0, 0 1, 1 1, 2 1, 2 0, 1 0, 0 0))

They represent the same polygon, but the vertex order has changed.

I think this is fine.

In addition, we want to make sure the behavior of ST_Union_Aggr is similar to PostGIS ST_Union (array variant): https://postgis.net/docs/ST_Union.html

[jiayuasu]

@zhangfengcdt Did you see performance improvement using this implementation, compared to the previous one?

[zhangfengcdt]

@zhangfengcdt Did you see performance improvement using this implementation, compared to the previous one?

Yeah, I am adding some tests to report the performance measure and we can see the improvements for different cases there.

[zhangfengcdt]

@zhangfengcdt Did you see performance improvement using this implementation, compared to the previous one?

@jiayuasu I have used the newly added test to measure both old and new runtime for different number of geometries. Here are the results:

 Number of Polygons   |  OLD ST_Union_Aggr (in ms) |  NEW ST_Union_Aggr (in ms)
 -------------------------------------------------------------------------------
 100                  |             297            |              354     
 500                  |             750            |              386
 1,000                |           2,231            |              430
 5,000                |          53,870            |            1,400
 10,000               |         243,465            |            3,474

I think it shows clearly the new method is much efficient and scalable.

[jiayuasu]

Can you return a reference of the DF as the return value of the function, instead of creating a new temp view? Otherwise this might pollute the global namespace and lead to bugs that are hard to find.

[jiayuasu]

Can you also unpersist this table at the end of the test case? Otherwise this will lead to memory leak.