Skip to content

Releases: AbsaOSS/spline-spark-agent

release/0.7.9

15 Jul 16:17
Compare
Choose a tag to compare

Features

  • #452 - Spark 3.3 support
  • #36 - ElasticSearch-hadoop connector support Scala 2.12
  • #53 - Add a fallback lineage dispatcher
  • #396 - HttpLineageDispatcher to not expect Spline server on the other end

Bugfixes

  • #450 - Fix: When Spark is instructed to create a View, Attribute lineage breaks

Tech debts

  • #443 - Fix Scala compiler warnings
  • #460 - Exclude test classes from code duplication check on Sonar

0.7.8

19 May 19:45
Compare
Choose a tag to compare

Features

  • #426 Iceberg support

Bugfixes

  • Fix handling errors occurred in the ObjectStructureDumper (#427, #137)
  • Fix java.lang.NumberFormatException: For input string: "1,146"… (#431)

0.7.7

25 Apr 11:12
Compare
Choose a tag to compare

Bugfixes

  • #423 Fix "json4s-ext" incompatibility for Spark 3.2

0.7.6 - Spark 3.2 support

07 Apr 09:33
Compare
Choose a tag to compare

Features

Bugfixes

  • #410 - fix compilation on case-insensitive file systems

0.7.4

21 Mar 14:25
Compare
Choose a tag to compare

Bugfixes

  • #408 - fix ObjectStructureDumper exception logging
  • #417 - fix timeout settings not propagated to the server

0.7.3

09 Feb 13:14
Compare
Choose a tag to compare

Bugfixes

#383 - fix Kafka dispatcher error Cannot perform operation after producer has been closed

0.7.2

22 Dec 17:19
Compare
Choose a tag to compare

Bugfixes

#370 - Fix Spark/Hadoop configuration string array properties parsing

0.7.1

04 Nov 12:48
Compare
Choose a tag to compare

Bugfixes

#346 - Unexpected error using spline agent 0.7 with AWS Glue 3.0

Improvements

#348 - Capture more details about object data types

0.7.0

18 Oct 22:38
Compare
Choose a tag to compare

Summary

This release includes two important improvements:

  1. The agent dependencies have been optimized. It resulted in 4 times decrease of the bundle JAR size - 3 Mb instead of 12 Mb (#91)
  2. Switching to UUID version 5 in combination with a few other changes allowed to make the agent output deterministic.It helps deduplicating lineage of recurring jobs which reduces the Spline database size and load on the server. (#323)

Bugfixes

#314 - KafkaDispatcher: set max.in.flight.requests.per.connection to 1 by default to avoid reordering
#328 - Lineage harvester output is not deterministic

Improvements

#91 - Optimize dependencies
#323 - Support for multiple types of ID
#326 - Logging: Spline init errors in BEST_EFFORT mode should be logged as WARN

Documentation

#297 - Document versioning strategy

0.6.4 - Java 11+ & Google BigQuery

16 Sep 15:08
Compare
Choose a tag to compare

Features

Bugfixes

  • #308 - Fix an error when running REPL under Java 11+