Feature request: Access MCAP attachments / metadata from panel context

[bgromov]

We store sensors configuration as a JSON file attachment in our MCAP files. That config is needed for correct visualization of various data, e.g. for transforming point clouds.

As a workaround, we have a button in the custom panel to open the same MCAP again (because there is no way to get the open MCAP filename from the panel context), but nothing prevents the user choosing a wrong file, so that solution is really suboptimal.

[defunctzombie]

Thanks for the request; it does make sense that this kind of information gets recorded in attachments. One general comment/advice I can give about workflows that de-couple this requirement. The decoupling becomes more useful as you scale up development and move data management thinking away from individual files towards data over time for various devices. This information does need to be fetched for the correct timespan and surfaced somehow for use in panels. I'm thinking we would leverage the existing asset interfaces we've started to test in some panels for getting at this information.

What kind of sensors configuration is stored and how does it apply to the visualization data? Is this in a custom panel that you visualize this data or need this applied to visualize in the built-in 3d panel?

[bgromov]

We have several use cases, among those are hardware-in-the-loop testing and fast prototyping of sensor fusion algorithms.

For hardware-in-the-loop, we capture the state of the device that produced the data and then use it to reconfigure the hardware when we feed that data back. Configs are installation specific because of sensor calibration etc. So, having this information in the same file with data is crucial, and I don't think it can be decoupled.

The device has several sensors connected to it and it knows their calibrated 6D poses, so it can process the sensor data and produce, e.g. odometry, but for debugging purposes it also spits out raw data (in sensor's frame), so we need these configs to make sense of the raw data when we develop our algorithms. Foxglove Studio seems like a perfect solution for that kind of prototyping / simulation but there are a few things missing to make it work. Access to the metadata of the replayed MCAP is one of them and being able to produce and publish new data is another (that's my other feature request #975).

Ideally, we don't need a separate panel for visualization but we do need the settings pane (to, e.g., play with calibration parameters and see how the system behaviour changes) and then we can visualize everything in the built-in 3D panel using standard primitives like point clouds etc.

What we really want to avoid is to have any other scripts / software running in parallel with Foxglove Studio (that would also kill the browser-based solution for us).