Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][Connector-v2][PulsarSink]Add Pulsar Sink Connector. #4382

Merged
merged 35 commits into from
Jan 11, 2024
Merged

[Feature][Connector-v2][PulsarSink]Add Pulsar Sink Connector. #4382

merged 35 commits into from
Jan 11, 2024

Conversation

lightzhao
Copy link
Contributor

@lightzhao lightzhao commented Mar 21, 2023 •
edited
Loading

Purpose of this pull request

Add Pulsar Sink Connector. #3018

Check list

@lightzhao
Copy link
Contributor Author

lightzhao commented Mar 24, 2023 •
edited
Loading

Hello everyone, ci check cannot automatically, ci check require approval?

@lightzhao
Copy link
Contributor Author

Hello everyone, it has been running stably in the test environment for two weeks, who has time to help review.

hailin0
hailin0 reviewed Mar 27, 2023
View reviewed changes
Copy link
Member

@hailin0 hailin0 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please update release-note.md
please add e2e testcase

reference
https://github.com/apache/incubator-seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e

hailin0
hailin0 reviewed Mar 27, 2023
View reviewed changes
docs/en/connector-v2/sink/pulsar.md Outdated
Comment on lines 11 to 12
- [x] [batch](../../concept/connector-v2-features.md)
- [x] [stream](../../concept/connector-v2-features.md)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [x] [batch](../../concept/connector-v2-features.md)
- [x] [stream](../../concept/connector-v2-features.md)

https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/concept/connector-v2-features.md#sink-connector-features

docs/en/connector-v2/sink/pulsar.md Outdated
Comment on lines 14 to 16
- [ ] [column projection](../../concept/connector-v2-features.md)
- [x] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [ ] [column projection](../../concept/connector-v2-features.md)
- [x] [parallelism](../../concept/connector-v2-features.md)
- [ ] [support user-defined split](../../concept/connector-v2-features.md)

@lightzhao
Copy link
Contributor Author

please update release-note.md please add e2e testcase

reference https://github.com/apache/incubator-seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e

waiting #3990 ,In this pr I added the pulsar-e2e module. doc has been updated.

@lightzhao
Copy link
Contributor Author

Hello everyone, why can't the automatic ci check be performed after updating the code now, and it needs to be approved?

@TyrantLucifer
Copy link
Member

Hello everyone, why can't the automatic ci check be performed after updating the code now, and it needs to be approved?

Yes, infra changed the rule of this. It needs to be approved.

@lightzhao
Copy link
Contributor Author

Hello everyone, why can't the automatic ci check be performed after updating the code now, and it needs to be approved?

Yes, infra changed the rule of this. It needs to be approved.

please review #3990，and then add pulsar sink e2e test. thanks.

@lightzhao lightzhao mentioned this pull request Apr 13, 2023
Closed
5 tasks
@lightzhao lightzhao mentioned this pull request May 12, 2023
Closed
3 tasks
@lightzhao lightzhao requested a review from hailin0 May 26, 2023 10:37
@lightzhao
Copy link
Contributor Author

please approve ci.

@lightzhao
Copy link
Contributor Author

@Hisoka-X @hailin0 @TyrantLucifer please approve ci.

@lightzhao
Copy link
Contributor Author

@Hisoka-X @hailin0 @EricJoy2048 please approve ci.

@lightzhao
Copy link
Contributor Author

@TyrantLucifer @hailin0 @EricJoy2048 @Hisoka-X PTAL.

@lightzhao
Copy link
Contributor Author

@ALL Can anyone help to review this pr?

@lightzhao
Copy link
Contributor Author

@TyrantLucifer @hailin0 @EricJoy2048 @Hisoka-X PTAL.

@lightzhao
Copy link
Contributor Author

It's been a long time, @TyrantLucifer @hailin0 @EricJoy2048 @Hisoka-X PTAL.

@ic4y ic4y closed this Aug 5, 2023
Carl-Zhou-CN
Carl-Zhou-CN reviewed Dec 19, 2023
View reviewed changes
...ar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/sink/PulsarSinkWriter.java Outdated
/**
* get message SerializationSchema
*
* @param rowType
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment of these parameters is useless, can we delete it？

Carl-Zhou-CN
Carl-Zhou-CN reviewed Dec 19, 2023
View reviewed changes
...ar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/sink/PulsarSinkWriter.java Outdated
/**
* get key SerializationSchema
*
* @param keyFieldNames
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Carl-Zhou-CN
Carl-Zhou-CN reviewed Dec 19, 2023
View reviewed changes
...ar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/sink/PulsarSinkWriter.java Outdated
/**
* get partition key field list
*
* @param pluginConfig
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@lightzhao
Copy link
Contributor Author

@hailin0 @Carl-Zhou-CN @Hisoka-X PTAL.

Carl-Zhou-CN
Carl-Zhou-CN previously approved these changes Dec 24, 2023
View reviewed changes
Copy link
Member

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

hailin0
hailin0 reviewed Dec 26, 2023
View reviewed changes
Copy link
Member

@hailin0 hailin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

...l-e2e/seatunnel-connector-v2-e2e/connector-pulsar-e2e/src/test/resources/fake_to_pulsar.conf Outdated Show resolved Hide resolved
@EricJoy2048 @hailin0
Update seatunnel-e2e/seatunnel-connector-v2-e2e/connector-pulsar-e2e/…
126874d 
…src/test/resources/fake_to_pulsar.conf

Co-authored-by: hailin0 <[email protected]>
@lightzhao
Copy link
Contributor Author

@hailin0 @Hisoka-X @Carl-Zhou-CN PTAL.

@hailin0 hailin0 merged commit 543d2c5 into apache:dev Jan 11, 2024
7 checks passed
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
@lightzhao
[Feature][Connector-v2][PulsarSink]Add Pulsar Sink Connector. (apache…
56825be 
…#4382)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@EricJoy2048 EricJoy2048 EricJoy2048 approved these changes

@hailin0 hailin0 hailin0 approved these changes

@Hisoka-X Hisoka-X Awaiting requested review from Hisoka-X

@Carl-Zhou-CN Carl-Zhou-CN Awaiting requested review from Carl-Zhou-CN

Assignees
No one assigned
Labels
approved reviewed
Projects
None yet
Milestone
2.3.4
Development

Successfully merging this pull request may close these issues.
7 participants
@lightzhao @TyrantLucifer @ic4y @EricJoy2048 @hailin0 @Hisoka-X @Carl-Zhou-CN