MySQL Binary Log connector.
Initially project was started as a fork of open-replicator, but ended up as a complete rewrite. Key differences/features:
- automatic binlog filename/position resolution
- resumable disconnects
- plugable failover strategies
- JMX exposure (optionally with statistics)
- availability in Maven Central
- no third-party dependencies
- binlog_checksum support (for MySQL 5.6.2+ users)
- test suite over different versions of MySQL releases
Get the latest JAR(s) from here. Alternatively you can include following Maven dependency (available through Maven Central):
<dependency>
<groupId>com.github.shyiko</groupId>
<artifactId>mysql-binlog-connector-java</artifactId>
<version>0.1.0</version>
</dependency>
The latest development version always available through Sonatype Snapshots repository (as shown below).
<dependencies>
<dependency>
<groupId>com.github.shyiko</groupId>
<artifactId>mysql-binlog-connector-java</artifactId>
<version>0.1.1-SNAPSHOT</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>sonatype-snapshots</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
File binlogFile = ...
BinaryLogFileReader reader = new BinaryLogFileReader(binlogFile);
try {
for (Event event; (event = reader.readEvent()) != null; ) {
...
}
} finally {
reader.close();
}
PREREQUISITES: Whichever user you plan to use for the BinaryLogClient, he MUST have REPLICATION SLAVE privilege. Unless you specify binlogFilename/binlogPosition yourself (in which case automatic resolution won't kick in), you'll need REPLICATION CLIENT granted as well.
BinaryLogClient client = new BinaryLogClient("hostname", 3306, "username", "password");
client.registerEventListener(new EventListener() {
@Override
public void onEvent(Event event) {
...
}
});
client.connect();
By default, BinaryLogClient starts from the current (at the time of connect) master binlog position. If you wish to kick off from a specific filename or position, use client.setBinlogFilename(...) + client.setBinlogPosition(...).
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
BinaryLogClient binaryLogClient = ...
ObjectName objectName = new ObjectName("mysql.binlog:type=BinaryLogClient");
mBeanServer.registerMBean(binaryLogClient, objectName);
// following bean accumulates various BinaryLogClient stats (e.g. number of disconnects, skipped events)
BinaryLogClientStatistics stats = new BinaryLogClientStatistics(binaryLogClient);
ObjectName statsObjectName = new ObjectName("mysql.binlog:type=BinaryLogClientStatistics");
mBeanServer.registerMBean(stats, statsObjectName);
- data of numeric types (tinyint, etc) always returned signed(!) regardless of whether column definition includes "unsigned" keyword or not
- data of *text/*blob types always returned as a byte array
- timestamp is timezone-sensitive
For the insight into the internals of MySQL look here. MySQL Client/Server Protocol and The Binary Log sections are particularly useful as a reference documentation for the com.**.mysql.binlog.network
and com.**.mysql.binlog.event
packages.
git clone https://github.com/shyiko/mysql-binlog-connector-java.git
cd mysql-binlog-connector-java
mvn # shows how to build, test, etc. project
- rook - Change Data Capture (CDC) toolkit for keeping system layers in sync with the database.
In lieu of a formal styleguide, please take care to maintain the existing coding style.
Executing mvn checkstyle:check
within project directory should not produce any errors.
If you are willing to install vagrant (required by integration tests) it's highly recommended
to check (with mvn clean verify
) that there are no test failures before sending a pull request.
Additional tests for any new or changed functionality are also very welcomed.