1.0.5

Bugfixes

• Fix java.lang.IllegalArgumentException: transpose in MergeIntoNodeBuilder, and improve attribute dependency resolving for MERGE INTO operation (#605)
• Fix YAML configuration parser (#610, #616)

1.0.4

Bugfixes

• remove transitive dependencies accidentally packed into the bundle jars since the version 1.0.2, that in some cases might cause classpath conflict and result in Spark misbehavior (#595, #587)

1.0.3

Bugfixes

#584 - Fix lineage capturing of the MERGE INTO command on Databricks.
#588 - Fix JSON output format of the console and logging dispatchers to be the same as for the http or kafka dispatchers.
#589 - Fix an incorrect apiVersion property value format in the default YAML config file.

1.0.2

Bugfixes

#579 - IncompatibleClassChangeError: class z.c.a.s.harvester.plugin.embedded.DeltaPlugin$SyntheticDeltaRead
#574 - Complex queries in MERGE INTO USING - lineage extraction doesn't work

Tech depts

#577 - Replace deprecated methods by toOption and toNonEmptyOption from commons
#581 - Improve integration tests to cover binary compatibility between the Spline core and target Spark versions

1.0.1

Bugfixes

• #571 - Fix missing expression details in operations like Projection, Sort, Filter etc.

1.0.0

Note: this is a new major 1.0.0 release.
Although we strive for backward compatibility even in major releases, in some areas (internals, extensions, configuration) it might introduce breaking changes.

New features

• Support for DELETE, UPDATE and MERGE operations in Delta (#517)
• Partial support for RDD operations (#498)
• Capture failed job executions (#32)
• Capture Spark user name (#474)
• Option to enable capturing of memory only actions like collect(), count() etc (#361)
• Support for user-defined labels to annotate output Execution Plan and Execution Event objects (#378)
• Declarative way for capturing extra metadata by user-defined rules (#169)
• OpenLineage API support (for uplodaing the lineage to) (#354)
• New API for Spline agent programmatic initialization (makes it easier to add extensions) (#359)

HTTP lineage dispatcher

• Option to disable SSL validation (#524)
• Option to add custom HTTP headers (#482)

Kafka lineage dispatcher

• Option to select older versions of output JSON models (improves backward compatibility with older lineage hubs) (#376)

Configuration

• YAML format support for config files (#559)
• Add a default composite filter for easy reusage in custom filters (#476)
• Make DataSourcePasswordReplacingFilter enabled by default (#373)

Bugfixes

• Docker container won't stop on Ctrl+C (#398)
• Exceptions are swallowed in AgentBOM.postProcessingFilter (#372)
• Make log level independent on spline mode (#330)
• Integration tests :: SparkFixture :: custom session config is spilled onto other sessions (#566)
• Integration tests :: fix tests failing on Apple M1 (#511)
• Integration tests :: fix tests failing on Windows (#536)
• Examples :: com.fasterxml.jackson.core.JsonParseException (#529)
• Examples :: userExtraMeta filter doesn't seem to be applied (#527)

Minor enhancements

• Simplify spline.mode configuration: replace REQUIRED and BEST_EFFORT with ENABLED (#250, #489)
• Make the docker image 20% smaller in size (#298)
• Change the "Command To Be Implemented" error into a warning (#332)
• Add Spline payload JSON Schema URLs to custom OpenLineage facets (#416)
• and some other tech depts

0.7.13

Bugfixes

• #511 - fix integration tests on M1
• #543 - fix exception with SQL query from Spark temporary view
• #542 - fix OneRowRelationFilter Option handling

0.7.12

Features

• Delta Lake 2.1.0 support (#508)
• Iceberg 1.0.0 Support (#509)

Bugfixes

• Fix java.lang.NoSuchFieldException for CreateTableAsSelect statements (#487)

0.7.11

Features

• #479 - Add support for Delta Lake 1.2
• #480 - Add support for Delta Lake 2.0

Improvements

• #488 - HttpLineageDispatcher to not call REST /status unnecessarily when all required properties (apiVersion and requestCompression) are specified in the dispatcher configuration

release/0.7.10

See release notes for the version 0.7.9

This release fixes examples and artifact publishing issue - #467