Filter Out Failed Blocks in grpc2kafka Configuration

[mohsen-vybenetwork]

Hi Team,

We just realized that we are receiving all blocks, including failed blocks with data. For example, in our Kafka, we have data for both of these blocks: 293553856 and 293553861, where 293553861 is a failed block. Is there a way to filter these out?

I noticed that we have a filter_by_commitment option in the configuration, but I'm unsure how to use it in the grpc2kafka JSON config. Could you provide guidance on how to set this up?

293553861 (failed block, not real)
293553856 (valid block, real)

This is my config for TX and I want to know is there a way to get only finalized blocks?

"request": {
            "blocks": {
                "client": {
                    "include_transactions":true,
                    "include_accounts": false
                }
 }

[fanatid]

grpc2kafka was moved to https://github.com/rpcpool/yellowstone-grpc-kafka

In addition to blocks you need to set commitment: "finalized", i.e. something like:

"request": {
            "blocks": {
                "client": {
                    "include_transactions":true,
                    "include_accounts": false
                },
"commitment": "finalized"
 }

but I'd not recommend stream blocks, sometimes we are not able to construct them in the plugin, stream finalized transactions instead of blocks would be the preferred way.

[mohsen-vybenetwork]

@fanatid Thank you for your response and advice. You are awesome. Could you please provide a sample config for the transactions stream with filtering for failed transactions? Does it include blocktime? The only reason we stream blocks is to have blocktime associated with the slot.

[fanatid]

No, transactions don't provide block time but you can subscribe on block meta too, and fill time after receiving block meta.