Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][Transform-v2] Add metadata transform #7899

Merged
merged 4 commits into from
Nov 11, 2024
Merged

[Feature][Transform-v2] Add metadata transform #7899

merged 4 commits into from
Nov 11, 2024

Conversation

hawk9821
Copy link
Contributor

@hawk9821 hawk9821 commented Oct 24, 2024 •
edited
Loading

Purpose of this pull request

add metadata transform, supported metadata fields :

  • Database
  • Table
  • Partition
  • RowKind
  • EventTime
  • Delay
    RowKind, EventTime, Delay use to debezium CDC Currently.

Does this PR introduce any user-facing change?

How was this patch tested?

UT: MetadataTransformTest#testMetadataTransform
E2E Case:

  • MysqlCDCIT#testMysqlCdcMetadataTrans
  • MongodbCDCIT#testMongodbCdcMetadataTrans
  • OpengaussCDCIT#testOpengaussCdcMeatadataTrans
  • OracleCDCIT#testOracleCdcMetadataTrans
  • PostgresCDCIT#testMPostgresCdcMetadataTrans
  • SqlServerCDCIT#testSqlServerCDCMetadataTrans

Check list

@hawk9821 hawk9821 force-pushed the metadata branch 2 times, most recently from 4838c1a to 6a603a0 Compare October 25, 2024 01:36
@hawk9821 hawk9821 force-pushed the metadata branch 3 times, most recently from a09c3fd to 4a4c5df Compare October 28, 2024 02:20
@hailin0 hailin0 mentioned this pull request Oct 28, 2024
Closed
3 tasks
@hawk9821 hawk9821 force-pushed the metadata branch 3 times, most recently from ddafc8f to 031cb7f Compare October 29, 2024 06:48
@hawk9821 hawk9821 force-pushed the metadata branch 2 times, most recently from c1c4312 to d130ec8 Compare October 30, 2024 16:09
Hisoka-X
Hisoka-X approved these changes Nov 11, 2024
View reviewed changes
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @hawk9821 !

@Hisoka-X Hisoka-X merged commit 699d165 into apache:dev Nov 11, 2024
5 checks passed
eyys pushed a commit to eyys/seatunnel that referenced this pull request Nov 12, 2024
@hawk9821 @eyys
[Feature][Transform-v2] Add metadata transform (apache#7899)
041e11d
@hawk9821 hawk9821 deleted the metadata branch November 13, 2024 01:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Hisoka-X Hisoka-X Hisoka-X approved these changes

@hailin0 hailin0 hailin0 approved these changes

Assignees
No one assigned
Labels
api approved cdc connectors-v2 document e2e reviewed transform-v2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hawk9821 @hailin0 @Hisoka-X