2.2.1

Bugfixes

• #844 - Fix missing functional expressions in the MERGE INTO ... operation lineage. Thanks to @AndreaCes

Full Changelog: release/2.2.0...release/2.2.1

2.2.0

Features

• #770 - Support for Spark 3.5
• #820 - Support for MaxCompute (thanks to @BerlinJava)

Bugfixes

• #821 - Fix incompatibility between different versions of Spark BigQuery Connector (thanks to @XavierZYXue)
• #827 - Fix Elasticsearch plugin to cover more variants of Elasticsearch Connector API usage.

2.1.0

What's Changed

• #705 - Added Suport for Spark 3.4 (thank you @rycowhi)
• #461 - Enable integration tests for Spark 3.3

Full Changelog: release/1.2.2...release/2.1.0

2.0.0

What's changed

This release is functionally the same as 1.2.2, with one subtle change in the extension API that makes it a braking change for those who is using custom Spline agent extensions (see below). Hence the major version bump.

In most cases just recompiling your extensions will be enough unless you explicitly used anything from the renamed package.

• #729 - Decouple from Absa Commons library and rename package za.co.absa.commons to za.co.absa.spline.commons

1.2.2

Bugfixes

• #723 - Fix an extension API breaking change that was introduced in version 1.2.1 (#715)

Improvements

• #726 - Simplify examples run script and docker image. Add multi-arch docker builds.

1.2.1

Bugfixes

• #714 - Add shading to agent bundles POMs to resolve a classpath collision on different versions of ABSA Commons library.

1.2.0

Bugfixes

• #623 - Fix an error related to incorrect closing the Kafka producer.
• #675 - Decouple from Spark embedded Json4s to fix sporadic classpath conflicts caused by inconsistent versions of Json4s and Jackson libraries provided by different versions and distributions of Spark (e.g. Databrics, EMR etc).
• #652 - Fix a reflection error in ExcelPlugin when reading from hdfs location.
• #700 - A hot fix for the breaking change in Spark internal API introduced as a remediation of the SPARK-43413 (Thank you @uday1409!).

Improvements

• #653 - The application name is now taken from the spark.app.name session configuration parameter, to allow for better customization of the names of jobs, that are executed within the same SparkContext instance (Thank you @uday1409).
• #649 - Developer setup improvement - add SDKMAN! config to automatically switch to the correct Java version required for the project build.

1.1.0

Features

• OAuth support for HTTP dispatchers (thanks to @rupesh3020) - #600, #618

Bugfixes

• Various fixes for AWS Glue - #602
• Fixes for Delta on Databricks - #570
• Fix NullPointerException thrown in some cases in the NonPersistentActionsCapturePlugin - #634
• Remove duplicated call to the close() method on Kafka producer in KafkaLineageDispatcher - #639

Other improvements

• Fix Docker build on M1 Apple silicon - #598
• Improve code formatting and similar minor tech debts - #607
• Added a note in the README about necessity to provide SnakeYAML as an additional dependency for Spline on Glue - #641

1.0.7

Enhancement

• Update Cobrix plugin to work with the recent Cobrix versions (#633)

1.0.6

Bugfixes

• Fix Symbols#CyclicReference exception thrown from ValueDecomposer in certain use cases - #625