Grafana Integration

[dwffls]

During @ct2034's talk at ROSCon 2024 the idea was started to visualize /diagnostics messages in a Grafana dashboard.
I wanted to restart this conversation here in the form of a feature request.
As I'd like to contribute to this feature I guess the first thing to tackle is the structure of this integration.

My own suggestion is to implement this in the diagnostics_aggregator and piggy back of the publishing of the /diagnostics_agg topic being send. This data would then be sent to Telegraf (taking inspiration from another talk at ROSCon) to be later used in grafana.

Happy to hear feedback!

[ct2034]

Hi
I think it is an interesting idea. Especially if you fine-tune your diagnostics information to contain all necessary status information, this could be a powerful tool for fleets.
And I also think that an aggregator is the right place to implement it. Then people can use the aggregator matching to choose the info to be piped to Grafana. I have not worked with Telegraf before, but it seems to be designed for these kinds of use cases.

[nnarain]

Hey guys. I'm also interested in seeing what can be done here to improve diagnostics.

My company has done an approach like this for many years (though not with telegraf/grafana but a similar stack). And we can visualize diagnostics metrics.

It might be worth discussing the future of rosdiagnostics and figuring out what the scope is. I'd personally think this could be implemented generically to handle any stack.

[dwffls]

@nnarain Could you please explain more how you would set it up to handle any stack? I guess any implementation (be that prometheus, telegraf or straight to influxdb) would need it's own configuration.

[nnarain]

So my take on it would be a new composable node that consumes the aggregated diagnostics topic and forwards it to the desired endpoint (telegraf, elastic, a network sockets, etc).

I personally wouldn't do this in the aggregator node as to not add new dependency for those that don't want to use a particular metrics stack.

Maybe something like "diagnostics_telegraf".

[dwffls]

Sending data to either InfluxDB itself or Telegraf works by sending a small http request, with the data formatted in a special text as such:

home,room=Living\ Room temp=21.1,hum=35.9,co=0i 1641024000
home,room=Kitchen temp=21.0,hum=35.9,co=0i 1641024000
home,room=Living\ Room temp=21.4,hum=35.9,co=0i 1641027600
home,room=Kitchen temp=23.0,hum=36.2,co=0i 1641027600
home,room=Living\ Room temp=21.8,hum=36.0,co=0i 1641031200

The only extra dependency we have to add to the aggregator node is curl. Personally I do not see this as a problem to include. @ct2034 What do you think?

[nnarain]

Ya I'd imagine a lot of these tools just use JSON.

So along the lines of what I mentioned earlier it might be a composable node that converts the DiagnosticArray into a JSON payload and sends it to an endpoint.

It sounds like a good use of composition to me. But it depends on what is and is not in scope of the aggregator