We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
java version "1.8.0_212" flink-1.17.1 hadoop3.3.4 下面三个jar都试过了 -rw-rw-r--. 1 liuqh liuqh 8452171 2月 23 17:28 flink-doris-connector-1.17-1.4.0.jar -rw-rw-r--. 1 liuqh liuqh 11915843 2月 23 17:28 flink-doris-connector-1.17-1.5.0.jar -rw-rw-r--. 1 liuqh liuqh 12193974 2月 23 17:28 flink-doris-connector-1.17-1.5.1.jar
在Standalone与Yarn Application模式下都能正常运行。但在Yarn Session模式下,数据同步存在问题。例如,当MySQL库中的test表有5条数据时,开启整库同步后,虽然成功创建了test表到Doris,但doris表数据为0条。随后对MySQL中的test表结构修改,表结构能够更新到Doris,但doris的test数据依然是0条
成功同步数据
`
bin/flink run -Dexecution.checkpointing.interval=10s -Dstate.checkpoints.dir=hdfs://hadoop102:8020/flink/checkpoints/report_sync -Dstate.savepoints.dir=hdfs://hadoop102:8020/flink/flink-savepoints/report_sync -Dparallelism.default=1 -c org.apache.doris.flink.tools.cdc.CdcTools job_jar/flink-doris-connector-1.17-1.5.0.jar mysql-sync-database --database test --mysql-conf server-id=5310-5313 --mysql-conf hostname=192.168.100.128 --mysql-conf port=3306 --mysql-conf username=root --mysql-conf password='123456' --mysql-conf database-name=test --sink-conf fenodes=192.168.88.120:8030 --sink-conf username=root --sink-conf password='Ryuu1234@' --sink-conf jdbc-url=jdbc:mysql://192.168.88.120:9030 --sink-conf sink.label-prefix=testx_sync2 --table-conf replication_num=1 --including-tables "test|test_tb01" `
No response
The text was updated successfully, but these errors were encountered:
You can check the configuration of jm and tm on flink web to see if the checkpoint and other information have been passed.
Sorry, something went wrong.
No branches or pull requests
Search before asking
Version
java version "1.8.0_212"
flink-1.17.1
hadoop3.3.4
下面三个jar都试过了
-rw-rw-r--. 1 liuqh liuqh 8452171 2月 23 17:28 flink-doris-connector-1.17-1.4.0.jar
-rw-rw-r--. 1 liuqh liuqh 11915843 2月 23 17:28 flink-doris-connector-1.17-1.5.0.jar
-rw-rw-r--. 1 liuqh liuqh 12193974 2月 23 17:28 flink-doris-connector-1.17-1.5.1.jar
What's Wrong?
在Standalone与Yarn Application模式下都能正常运行。但在Yarn Session模式下,数据同步存在问题。例如,当MySQL库中的test表有5条数据时,开启整库同步后,虽然成功创建了test表到Doris,但doris表数据为0条。随后对MySQL中的test表结构修改,表结构能够更新到Doris,但doris的test数据依然是0条
What You Expected?
成功同步数据
How to Reproduce?
`
1.5.0yarn-session没办法删除数据,但是可以同步表结构
bin/flink run
-Dexecution.checkpointing.interval=10s
-Dstate.checkpoints.dir=hdfs://hadoop102:8020/flink/checkpoints/report_sync
-Dstate.savepoints.dir=hdfs://hadoop102:8020/flink/flink-savepoints/report_sync
-Dparallelism.default=1
-c org.apache.doris.flink.tools.cdc.CdcTools
job_jar/flink-doris-connector-1.17-1.5.0.jar
mysql-sync-database
--database test
--mysql-conf
server-id=5310-5313
--mysql-conf hostname=192.168.100.128
--mysql-conf port=3306
--mysql-conf username=root
--mysql-conf password='123456'
--mysql-conf database-name=test
--sink-conf fenodes=192.168.88.120:8030
--sink-conf username=root
--sink-conf password='Ryuu1234@'
--sink-conf jdbc-url=jdbc:mysql://192.168.88.120:9030
--sink-conf sink.label-prefix=testx_sync2
--table-conf replication_num=1
--including-tables "test|test_tb01"
`
Anything Else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: