Releases: AbsaOSS/spline-spark-agent
Releases · AbsaOSS/spline-spark-agent
release/0.7.9
Features
- #452 - Spark 3.3 support
- #36 - ElasticSearch-hadoop connector support Scala 2.12
- #53 - Add a fallback lineage dispatcher
- #396 - HttpLineageDispatcher to not expect Spline server on the other end
Bugfixes
- #450 - Fix: When Spark is instructed to create a View, Attribute lineage breaks
Tech debts
0.7.8
0.7.7
0.7.6 - Spark 3.2 support
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
Summary
This release includes two important improvements:
- The agent dependencies have been optimized. It resulted in 4 times decrease of the bundle JAR size - 3 Mb instead of 12 Mb (#91)
- Switching to UUID version 5 in combination with a few other changes allowed to make the agent output deterministic.It helps deduplicating lineage of recurring jobs which reduces the Spline database size and load on the server. (#323)
Bugfixes
#314 - KafkaDispatcher: set max.in.flight.requests.per.connection to 1 by default to avoid reordering
#328 - Lineage harvester output is not deterministic
Improvements
#91 - Optimize dependencies
#323 - Support for multiple types of ID
#326 - Logging: Spline init errors in BEST_EFFORT mode should be logged as WARN
Documentation
#297 - Document versioning strategy
0.6.4 - Java 11+ & Google BigQuery
Features
- #277 - Google BigQuery support (Thanks @mohanajuhi166)
Bugfixes
- #308 - Fix an error when running REPL under Java 11+