Plotting real-time JSON stream from another application

[pavel-kirienko]

First things first: I am new to PJ and even though I did complete the tutorial and done my homework RTFMing I might have missed something, in which case any pointers would be appreciated.

I have a tool similar to rostopic echo that consumes data from a pub-sub network and prints that in JSON. I would like to visualize the data in real-time, and oddly, there are very few solutions available for this. With PlotJuggler I ended up using the UDP server mode while piping the stdout of my tool into nc -u localhost 9870 which seems to work well but the downside is that it is unergonomic, as you can't just type one short command to plot the data of interest from a live system, you have to set up multiple things and remember the UDP port number. Would I be the first to ask for the ability to accept JSON/whatever via stdin? In my case specifically (but I suspect I am not the only one here), instead of doing this:

# (launch an instance of PlotJuggler)
# (start the UDP server)
$ yakut subscribe <topics topics> | nc -u localhost 9870

I could go like:

$ yakut subscribe <topics topics> | pj --stdin-json-or-something

(I did consider using Gnuplot for this but it is not really usable in this scenario for more than one reason)

Then it would also be useful for quickly visualizing JSON dumps without having to click through the GUI, like:

$ cat dump.json | pj --stdin-json-or-something

Here's another thing: why is the buffer span limited to 100 seconds? Say I want to collect a large time series from a live system and I don't really care about memory usage and I can tolerate some sluggishness of the UI if it comes to that. Why is it not possible to set this to 10000?:

Here's the last observation: the tool supports CSV which is great but how about TSV? Seems like a trivial thing to support. I have a certain test rig that produces reports in TSV which I currently visualize using Jupyter+Pandas but PJ seems like it could be a better option if it was able to accept TSV without preprocessing.

Am I trying to do weird things here or maybe missing something?

Cheers!

[facontidavide]

Would I be the first to ask for the ability to accept JSON/whatever via stdin

yes, you would 😄
PlotJuggler can use any inter-process communication you implement as a plugin. Pick one or implement one.

Here's another thing: why is the buffer span limited to 100 seconds?

Here's the last observation: the tool supports CSV which is great but how about TSV?

I didn't even know what TSV is! Yes, it is probably easy, since the CSV already support custom separator

[pavel-kirienko]

Here's another thing: why is the buffer span limited to 100 seconds?

Okay, then the next obvious question is: what use case is served by setting the default limit to such a low value?

[facontidavide]

PlotJuggler streaming is not supposed to log for a long time.
Some plugin save raw data in the buffer and this raw data can be quite big, even if it is not displayed.

This may not affect YOU, but may affect others. Anyway, this is a design decision that I don't want to change for reasons that are not necessarily obvious