v0.11.0

Schema Support:

• Introduced comprehensive schema support, allowing schemas to be attached to records.
• Integrated a built-in schema registry, with the option to use an external schema registry.
• Enhanced standalone connectors to retain type information.

Connector Improvements:

• Introduced middleware for automatic schema handling in built-in connectors and processors.
• Postgres Connector:
  • Added support for incremental snapshots in logical replication mode.
  • Improved management of logical replication slots.

Processor Improvements:

• error
  • Introduced an error processor to send records to DLQ or fail the pipeline based on conditions.
• webhook.http
  • Added the ability to specify headers in the webhook.http processor.
• field.convert
  • Extended to support conversion to Go time.Time objects, handling unix nano timestamps and RFC3339 formatted dates.
• avro.encode and avro.decode
  • Updated to use Conduit’s built-in schema registry.

Internal Improvements:

• Fixed bug causing a panic when trying to run a partial pipeline.

Changelog

• ab0ecac: Mention storage options in README (#1730) (@hariso)
• 428e651: Database implementation for SQLite3 (#1682) (@lyuboxa)
• bf9b0c7: Include conduit-schema-registry in release instructions (#1713) (@hariso)
• 840a650: More info about Postgres connection string (#1726) (@hariso)
• 7339241: Replace panic with error message when not declaring a source node in .yml config (#1695) (@AdamHaffar)
• 1397d5d: Replace pkg/foundation/database with commons, use schema registry with DB (#1720) (@hariso)
• 930afae: Schema support (#1702) (@lovromazgon)
• 3a9d244: Tests and examples for converting time using field.convert (#1745) (@lovromazgon)
• 5c956e1: Update SDK, protocol, Kafka connector (#1731) (@hariso)
• 7601e06: Update dependencies for v0.11.0 (#1756) (@lovromazgon)
• f53d710: Update linter config (#1675) (@hariso)
• d6e83ce: Update processor SDK, add middleware (#1742) (@lovromazgon)
• 06bd6e5: Use connector protocol v2 (#1622) (@lovromazgon)
• 0d5929d: Use gofumpt (#1741) (@hariso)
• d370cd5: Use interpreter runtime in wasm tests (#1746) (@lovromazgon)
• 306defe: Wrap builtin processors in middleware in examples (#1748) (@lovromazgon)
• 76c25f1: [Schemas] Handle name conflicts (#1718) (@hariso)
• 203aeb7: [Schemas] Limit schema service usage to connectors (#1701) (@hariso)
• 817d52c: [Schemas] Use configured schema service in Avro processors (#1743) (@hariso)
• a9812b7: add getSchema and createSchema for wasm processors (#1683) (@maha-hajja)
• 76151f4: change keep-n-most-recent to cut-off (required parameter) (#1689) (@lovromazgon)
• 6b9dfa2: doc: update releases template (#1749) (@raulb)
• fd9f261: don't release win/i386 (#1703) (@lovromazgon)
• 7bb8cd8: enable container-retention-policy action (#1693) (@lovromazgon)
• a8d76dc: feat: sets environment variable based on log level (#1740) (@raulb)
• 2da3a0b: refactor: architecture documentation (#1732) (@raulb)
• b6fa5be: support empty .payload.After / .payload.Before in avro.decode (#1678) (@samirketema)
• fab3ab9: update swagger ui (#1711) (@lovromazgon)

Release Artifacts

OSX Darwin

• 📦 conduit_0.11.0_Darwin_arm64.tar.gz
• 📦 conduit_0.11.0_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.11.0_Linux_arm64.deb
• 📦 conduit_0.11.0_Linux_arm64.rpm
• 📦 conduit_0.11.0_Linux_arm64.tar.gz
• 📦 conduit_0.11.0_Linux_i386.deb
• 📦 conduit_0.11.0_Linux_i386.rpm
• 📦 conduit_0.11.0_Linux_i386.tar.gz
• 📦 conduit_0.11.0_Linux_x86_64.deb
• 📦 conduit_0.11.0_Linux_x86_64.rpm
• 📦 conduit_0.11.0_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.11.0_Windows_arm64.zip
• 📦 conduit_0.11.0_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.11.0