Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kafka-connect Build a Kafka Source connector to pull Adobe IO Events and stream these in Kafka #159

Open
francoisledroff opened this issue Jun 1, 2023 · 1 comment
Open

kafka-connect Build a Kafka Source connector to pull Adobe IO Events and stream these in Kafka #159

francoisledroff opened this issue Jun 1, 2023 · 1 comment

Comments

@francoisledroff
Copy link
Collaborator

Build a Kafka Source kafka-connect connector :

  • that leverages aio-lib-java sdk to pull Adobe IO Events from the journaling API
  • and stream these Events in Kafka.

see https://kafka.apache.org/documentation/#connect
@OneCricketeer
Copy link

OneCricketeer commented Jun 6, 2023 •
edited
Loading

There already exists a sink connector. Perhaps add the source connector class to that?

One concern I had with the journalling API, however was that it presents a "single partitioned" endpoint, so something to either consider for Kafka consumer scalability, or to document that connector producer should write to a single partitioned topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@francoisledroff @OneCricketeer