Fix logging rates #146
Fix logging rates #146
Conversation
Signed-off-by: Michael Anderson <[email protected]>
Signed-off-by: Michael Anderson <[email protected]>
Signed-off-by: Michael Anderson <[email protected]>
|
@hamilton8415 using this branch, I can verify that the pub rates increase with RTF relative to wall, but stay 10Hz relative to sim clock |
Signed-off-by: Michael Anderson <[email protected]>
|
Getting there! I did some runs with this and it's so nice to be able to run fast, really opens up what's possible... It looks like perhaps the buoy AHRS (DataID = 3) has been overlooked? I plotted the time and time-between samples in the log files and everything lines up perfectly on 0.1seconds (when running faster than real-time), but the ^3 lines are still lagging. These are denoted "XB" in the plot below.
|
|
And in another interesting plot, I change the pack-rate for the spring controller and the power converter using pbcmd while running at faster than realtime. For the most part the output behaves as can be seen in this plot.
The basic speed of the packets changes for sure, but one can see some issues with dither in the timesteps when the packet rate is at the highest speed (50Hz), it still averages a timestep of 0.02, but it's sometimes zero, and sometimes faster. Looking at the .csv files, there is sufficent resolution in the timestamps (3 digits). |
|
And finally, pbcmd is not exiting gracefully with these pack commands, they work fine though... hamilton@WOLFPACK:~$ pc_PackRate 25 |
|
Can you repeat your plots for running specific RTF's? Can you try |
|
I'm pretty certain it comes down to the physics step. If you'd like to have data available every 0.02 seconds, but our physics step is 0.1... We are probably going to have some zero-order-hold going on. I was letting gz-sim decide the timestamp when it updates data, and then the publisher would use that timestamp, but I could hide the ZOH and have the publisher set the stamp when it sends and we'd then have the correct dt in the logs... |
|
I think that is the same issue with the AHRS. The simulated IMU sensor plugin is set to provide data at 50Hz to ensure precise data is available to the AHRS controller when it asks for it to publish at 10Hz which isn't possible with a timestep of 0.1 |
|
Can you repeat the test using a physics step of 0.02? |
|
As for the pbcmd.... uhhggghghhhghghg |
|
TC should be in the same boat as XB but I'm having trouble seeing those datapoints |
|
OK, I will check all of this. Does the RTF take an array of arguments and then run a number of runs? |
|
Yes, there is a chance that I ran the physics step slower than 50Hz for this test. Unfort I deleted the results but will re-run. makes sense...
_ |
No, that's one of the globals out of the matrix |
|
Here's an updated plot with the physics timestep at 0.01 and a variety of packet rates set via pbcmd. Still a bit of noise in there, but acceptable. Our typical matlab processing includes a step in which all of the controllers data points are interpolated onto a common time-base to facilitate analysis. And that worked fine... Also, TC data seems fine, just the XB (^3) is not keeping up...
|
|
Oh yes, sorry for the red-herring, my bad to ask it to give data more often than it was generating data... |
|
the dither there still isn't very satisfying... I'm guessing there's a race between the publish thread grabbing data and the simulation thread updating the data. I wasn't ever too happy with the low-/high-priority triple mutex thing. If it becomes a problem, I could look into a simpler/robust solution. |
Signed-off-by: Michael Anderson <[email protected]>
Signed-off-by: Michael Anderson <[email protected]>
|
@hamilton8415 can you run your plots for this commit? I think I may have fixed the AHRS pub rate |
|
Stand by, running now... |
|
Seemingly unchanged for me, but we should probably do a little branch checking. I'm on fix_logging_rate here and add_pblog_args for _utils... 3, 1684202088.468, ,,,,,,,,,,,,,,,,,,,,,,,0.285, 0.637, 28.947, 0.002, 0.003, 0.070, -0.195, 0.109, 10.111, -0.108, 0.200, -0.108, 0.00000, 0.00000, 0.000, 0.000, ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, Also, long buffer times when running RTF>1 present, here on the local machine too... |
|
you should be on @rhenthorn 's branch for _utils |
|
uhhh oops. I definitely found the problem this time. I never enabled the |
Signed-off-by: Michael Anderson <[email protected]>
|
the fix is in! |
|
Indeed, looking good. |
* add support for pblog Signed-off-by: Michael Anderson <[email protected]> * change default pblog root; add latest_rosbag symlink Signed-off-by: Michael Anderson <[email protected]> * Fixed issue by which wave-random seed wasn't being passed to Incident wave generator * Fix logging rates (#146) * add sim-time-aware Rate to controllers for publishing Signed-off-by: Michael Anderson <[email protected]> * forgot shutdowns to stop hanging Signed-off-by: Michael Anderson <[email protected]> * add copyright Signed-off-by: Michael Anderson <[email protected]> * linters Signed-off-by: Michael Anderson <[email protected]> * add async proxy to await future for pack rate calls (#147) Signed-off-by: Michael Anderson <[email protected]> * use sim time from postupdate instead of imu time Signed-off-by: Michael Anderson <[email protected]> * enable use_sim_time for AHRS Signed-off-by: Michael Anderson <[email protected]> --------- Signed-off-by: Michael Anderson <[email protected]> * linters Signed-off-by: Michael Anderson <[email protected]> * moved tf sensors to heavecone link; rotated imu to match physical Signed-off-by: Michael Anderson <[email protected]> * add gps Signed-off-by: Michael Anderson <[email protected]> --------- Signed-off-by: Michael Anderson <[email protected]> Co-authored-by: Andrew Hamilton <[email protected]>
add sim-time-aware Rate class for controlling publishing in all controllers