Motivation

The kafka-connect-elastic-search-sink allows you to import data from any Kafka topic into Elasticsearch indexes using the Kafka Connect API.

Using the Kafka Connect API, it is possible to stream data between Apache Kafka and other systems using a scalable and reliable tool. Kafka Connect features include:

• A common framework for Kafka connectors
• Distributed and standalone modes
• REST interface
• Automatic offset management
• Distributed and scalable by default
• Streaming/batch integration

All the features of Kafka Connect, such as offset management and fault tolerance, work with the kafka-connect-elastic-search-sink. You can restart and kill the processes and they will pick up where they left off, copying only new data.