-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: Update conduit internals to use conduit-commons #1433
Comments
simonl2002
added
housekeeping
Small improvements or chores
and removed
feature
New feature or request
labels
Mar 12, 2024
This was referenced May 14, 2024
This was referenced May 21, 2024
Migration script for connectors regarding the changes done in this issue: sed -i '' 's/conduit-connector-sdk\/cmd\/paramgen/conduit-commons\/paramgen/g' tools.go
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Record/opencdc.Record/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Operation/opencdc.Operation/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Position/opencdc.Position/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Change/opencdc.Change/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Data/opencdc.Data/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.RawData/opencdc.RawData/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.StructuredData/opencdc.StructuredData/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Metadata/opencdc.Metadata/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Parameter/config.Parameter/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Validation/config.Validation/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/Parameters() map\[string\]config.Parameter/Parameters() config.Parameters/g' {} +
find . -type f -iname "*.go" -exec sed -i '' '/ *Source) Configure(/ s/map\[string\]string/config.Config/g' {} +
find . -type f -iname "*.go" -exec sed -i '' '/ *Destination) Configure(/ s/map\[string\]string/config.Config/g' {} +
find . -type f -iname "*.go" -exec sed -i '' 's/sdk.Util.ParseConfig(/sdk.Util.ParseConfig(ctx, /g' {} +
goimports -w .
gci write .
go mod tidy
make generate Afterwards, try running tests and fix any issues that still exist. Before pushing the changes, also run |
57 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Feature description
This work includes removing config parameters and validations in the connector SDK in favor of using the config parameters in conduit-commons (see ConduitIO/conduit-commons#18 and ConduitIO/conduit-commons#19). This allows us to reuse the same code in connectors and processors. Also paramgen can then be removed from the SDK in favor of paramgen in conduit-commons, which will then work for both connectors and processors.
Additionally we need to remove
sdk.Record
(connector SDK) andrecord.Record
(Conduit) in favor ofopencdc.Record
in conduit-commons. This will simplify our codebase as well as have a positive impact on performance, since we won't have to do as many transformations between the types, which is currently happening in the hot path (for every record).The text was updated successfully, but these errors were encountered: