bug(source): unexpected warning on undefined-field-padding-null error

[xiangjinwu]

Describe the bug

In kafka source with upsert format, it is a normal case where a message only has key but no value. This represents a delete event.

However, when ingesting such a message, a warning is printed:

WARN rw-streaming actor{otel.name="Actor 24" actor_id=24 prev_epoch=6246912290127872 curr_epoch=6246912308543488}:executor{otel.name="Source 1800002716"}:source_parse_chunk{actor_id=24 source_id=2}:
risingwave_connector::parser: failed to parse non-pk column, padding with `NULL` error=Undefined field `value` at `` split_id="0" offset="6" column="first_value_column_name"

Error message/log

No response

To Reproduce

Just create a kafka upsert source, with a delete event (kafka message with key only and no value) in the topic.

Expected behavior

No warning printed. Because it is expected to have no value. It is misleading and may shadow/suppress other real problems.

When we already know it is a delete event, there should be no attempts to extract columns from its value.

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

delete calls into do_action

risingwave/src/connector/src/parser/mod.rs

Lines 536 to 541 in ff862dc

	pub fn delete(
	&mut self,
	f: impl FnMut(&SourceColumnDesc) -> AccessResult<Datum>,
	) -> AccessResult<()> {
	self.do_action::<OpActionDelete>(f)
	}

do_action calls wrapped_f (field access) to on every column self.descs.iter()

risingwave/src/connector/src/parser/mod.rs

Lines 498 to 505 in ff862dc

	let result = (self.descs.iter())
	.zip_eq_fast(self.builders.iter_mut())
	.try_for_each(|(desc, builder)| {
	wrapped_f(desc).map(|output| {
	A::apply(builder, output);
	applied_columns.push(builder);
	})
	});

[tabVersion]

it may involve compatibility issue for users using upsert format before we introducing include clause. May need further investigation.

[BugenZhao]

May related to #15525.

[xiangjinwu]

May related to #15525.

Related (same symptom) but need different solutions (different cause):

• That issue is to specify a subset of columns not to trigger warnings. This needs user input.
• This one is to avoid a subset of rows (delete ops) not to trigger warnings. This is purely internal and do less unnecessary work.

[github-actions]

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

[CAJan93]

FYI: A customer was confused by these log messages. IMHO if we have time and capacity we should fix the issue.