[Fast Data] Import a projection ignoring some specific field

[silversoul93]

What product are you having troubles with?

Console

What Console version are you using?

v12.3.0

Description

Hi guys,
we have a particular situation on the Fast Data involving:

• Oracle DB
• Debezium as CDC
• Fast Data event-driven architecture

The problem involves some CLOB fields on Oracle.
Since the CLOB fields are usually big (CHARACTER LARGE OBJECT) Debezium sends the CLOB fields only when they are inserted or updated and, if the fields aren't updated, Debezium sends "__debezium_unavailable_value" instead of sending the real value every time.

How can we, using the Fast Data, ignore the CLOB fields if Debezium send "__debezium_unavailable_value"?
Is it possible to use a cast function to ignore these fields if they contain "__debezium_unavailable_value"?

The only idea we had at the moment is to write a Custom Message Adapter, but it's overkill imho.

Thanks.

Actual Outcome

The projection is updated and the CLOB fields, on mongodb, contain the value "__debezium_unavailable_value".

Expected Outcome

The projection is updated without updating the CLOB fields if they contain the value "__debezium_unavailable_value".

[silversoul93]

Self update.
As discussed with @danibix95, the only way is to:

• write a custom adapter
• remove, from the custom adapter, all fields containing __debezium_unavailable_value from the before and the after JSON → in this way the importer will not override the field already saved on MongoDB, preserving the old value.

Moreover, the CLOB handling is still an alpha feature on Debezium, as reported in the official documentation.