Skip to content

1.0.0
Compare
Choose a tag to compare
@wajda wajda released this 30 Dec 11:19
· 129 commits to develop since this release
release/1.0.0
1e7455b

Note: this is a new major 1.0.0 release.
Although we strive for backward compatibility even in major releases, in some areas (internals, extensions, configuration) it might introduce breaking changes.

New features

  • Support for DELETE, UPDATE and MERGE operations in Delta (#517)
  • Partial support for RDD operations (#498)
  • Capture failed job executions (#32)
  • Capture Spark user name (#474)
  • Option to enable capturing of memory only actions like collect(), count() etc (#361)
  • Support for user-defined labels to annotate output Execution Plan and Execution Event objects (#378)
  • Declarative way for capturing extra metadata by user-defined rules (#169)
  • OpenLineage API support (for uplodaing the lineage to) (#354)
  • New API for Spline agent programmatic initialization (makes it easier to add extensions) (#359)

HTTP lineage dispatcher

  • Option to disable SSL validation (#524)
  • Option to add custom HTTP headers (#482)

Kafka lineage dispatcher

  • Option to select older versions of output JSON models (improves backward compatibility with older lineage hubs) (#376)

Configuration

  • YAML format support for config files (#559)
  • Add a default composite filter for easy reusage in custom filters (#476)
  • Make DataSourcePasswordReplacingFilter enabled by default (#373)

Bugfixes

  • Docker container won't stop on Ctrl+C (#398)
  • Exceptions are swallowed in AgentBOM.postProcessingFilter (#372)
  • Make log level independent on spline mode (#330)
  • Integration tests :: SparkFixture :: custom session config is spilled onto other sessions (#566)
  • Integration tests :: fix tests failing on Apple M1 (#511)
  • Integration tests :: fix tests failing on Windows (#536)
  • Examples :: com.fasterxml.jackson.core.JsonParseException (#529)
  • Examples :: userExtraMeta filter doesn't seem to be applied (#527)

Minor enhancements

  • Simplify spline.mode configuration: replace REQUIRED and BEST_EFFORT with ENABLED (#250, #489)
  • Make the docker image 20% smaller in size (#298)
  • Change the "Command To Be Implemented" error into a warning (#332)
  • Add Spline payload JSON Schema URLs to custom OpenLineage facets (#416)
  • and some other tech depts
Assets 2
Loading