Investigate Event Subscription integration in Tracelistener

[akc2267]

Tracelistener currently has to be colocated with a node to read traces from its KV store. As we move towards a more scalable and reliable architecture with multiple third-party failover nodes, we need to rethinking this strict coupling of tracelistener and a chain node.

DoD:

• investigate replacing the KV store traces with Event subscription https://docs.tendermint.com/master/tendermint-core/subscription.html
• investigate if any prominent chains have removed event emissions that we would need to update account and staking balances
• if this integration is feasible, create a high-level list of steps we must do to move to this new architecture
• write up your findings and share with the team on slack

[Pitasi]

In the Tendermint docs it says we have two different ways of getting events. The websocket one is being deprecated in Tendermint 0.36, but cosmos-sdk still is on Tendermint 0.35 so we don't really have a choice, we must use the websockets.

I did a simple test like this with crescent in staging:

# Forward port from crescent to my computer
kubectl port-forward svc/crescent 26657:26657

# Connect to it using a tool called "websocat"
websocat ws://127.0.0.1:26657/websocket

# Send a subscription query 
{ "jsonrpc": "2.0", "method": "subscribe", "id": 1, "params": { "query": "tm.event='NewBlock'" } }

And I got the new block event as requested, attached a file (pretty printed) here: https://app.zenhub.com/files/353632561/dd174941-9eb6-413b-a78e-df7fb7fe2c96/download

[akc2267]

maybe I'm misunderstanding, but I think this means we must use pure RPC going forward, and not rely on websockets.
https://docs.tendermint.com/master/tendermint-core/subscription.html#event-log-api

[Pitasi]

yes, but no one is using this tendermint core version yet

[Pitasi]

Anyway, from a quick look at that json it seems to me that this kind of data is "low level". What I mean is that this json file is (probably) the input for the SDK modules, and the modules can contain other logics to update the underlying application database of the chain 🤔

We can't reimplement the sdk modules logic ourselves, but maybe these events can be used as heuristic to update our "smart RPC cache" though.