[Bug] Using partial updated table array type field, when the array containing free value, insert the data error: Java. Lang. ArrayIndexOutOfBoundsException #4046
Comments
|
Hello, Could you please assign it to me? |
|
@dsanww it can't be reproduced in my pc with flink 1.18 and paimon 0.9 compiled by master.It completed successfully, and the data is correct by query.Could you try it once more? |
@ArthurSXL8 I tried to use the new version again（paimon-flink-1.18-0.9-20240826.002332-69.jar）, but still an error will be reported. To be more specific, after inserting the null row type data, the subsequent insertion of non-null data, the data did not change again. I checked flink web ui and found that the insertion error will be reported： error track： sql client query：
Flink SQL> select * from Flink SQL> insert into
Flink SQL> select * from Flink SQL> select * from |
@dsanww reproduced now, digging... |
|
@ArthurSXL8 @dsanww Do you have a more detailed exception stack trace? It seems we are missing a crucial part of the error stack. |
@JingsongLi This is the log file for the taskmanager |
|
@dsanww Problem solved? |
Not yet |
Search before asking
Paimon version
paimon-flink-1.18-0.9-20240708.002315-31.jar
Compute Engine
flink1.18
Minimal reproduce step
step1：
create TABLE if not exists
default.result_t(idint NOT NULL COMMENT '主键，全局唯一id',sexint ,join_infosarray<row<id int,name STRING,age INT>> COMMENT '品类id',update_timeTIMESTAMP(0) COMMENT '更新时间',PRIMARY KEY (
id) NOT ENFORCED)
WITH
(
'bucket' = '1',
'changelog-producer' = 'lookup',
'merge-engine'='partial-update',
'fields.update_time.sequence-group'='join_infos',
'fields.join_infos.aggregate-function' = 'nested_update',
'fields.join_infos.nested-key' = 'id'
);
step2:
insert into
default.result_tselect
6,
cast(null as int),
array[cast(null as row<id int,name STRING,age INT>)],
PROCTIME ();
The insert statement throws an exception：
insert into
default.result_tselect
6,
cast(null as int),
array[row(1,'2',3)],
PROCTIME ();
excpetion：
java.io.IOException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.paimon.flink.sink.StoreSinkWriteImpl.prepareCommit(StoreSinkWriteImpl.java:225)
at org.apache.paimon.flink.sink.TableWriteOperator.prepareCommit(TableWriteOperator.java:121)
at org.apache.paimon.flink.sink.RowDataStoreWriteOperator.prepareCommit(RowDataStoreWriteOperator.java:189)
at org.apache.paimon.flink.sink.PrepareCommitOperator.emitCommittables(PrepareCommitOperator.java:100)
at org.apache.paimon.flink.sink.PrepareCommitOperator.endInput(PrepareCommitOperator.java:88)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
What doesn't meet your expectations?
no error
Anything else?
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: