Fix joint_state_broadcaster performance issues #1640

jordan-palacios · 2025-04-16T07:43:37Z

This PR will include adding a new test for checking performance in a setup with lots of state interfaces (>500). I'll include here the results of each incremental performance fix commit added.

Since we have also noticed a lot of variance in the results, we'll run it a few times. All results will be summarized here.

Fixes #1409.

jordan-palacios · 2025-04-16T12:03:02Z

Performance of the current master version.

[INFO] [1744704812.699545539] [joint_state_broadcaster]: Average update time: 240.174700 us
[INFO] [1744704812.699598892] [joint_state_broadcaster]: Variance: 231730.945780 us
--
[INFO] [1744704815.285795981] [joint_state_broadcaster]: Average update time: 236.002800 us
[INFO] [1744704815.285851910] [joint_state_broadcaster]: Variance: 297345.111392 us
--
[INFO] [1744704817.849313397] [joint_state_broadcaster]: Average update time: 234.468100 us
[INFO] [1744704817.849375057] [joint_state_broadcaster]: Variance: 231589.582582 us
--
[INFO] [1744704820.471171285] [joint_state_broadcaster]: Average update time: 239.791200 us
[INFO] [1744704820.471218683] [joint_state_broadcaster]: Variance: 462334.572003 us
--
[INFO] [1744704823.045886822] [joint_state_broadcaster]: Average update time: 235.594400 us
[INFO] [1744704823.045938966] [joint_state_broadcaster]: Variance: 26580.964489 us
--
[INFO] [1744704825.625277357] [joint_state_broadcaster]: Average update time: 235.583400 us
[INFO] [1744704825.625330096] [joint_state_broadcaster]: Variance: 231611.694044 us
--
[INFO] [1744704828.254401907] [joint_state_broadcaster]: Average update time: 241.029100 us
[INFO] [1744704828.254448233] [joint_state_broadcaster]: Variance: 462391.610853 us
--
[INFO] [1744704830.802185520] [joint_state_broadcaster]: Average update time: 232.373100 us
[INFO] [1744704830.802232534] [joint_state_broadcaster]: Variance: 811.116296 us
--
[INFO] [1744704833.448850699] [joint_state_broadcaster]: Average update time: 242.778300 us
[INFO] [1744704833.448903136] [joint_state_broadcaster]: Variance: 462345.285149 us
--
[INFO] [1744704836.060410410] [joint_state_broadcaster]: Average update time: 238.798000 us
[INFO] [1744704836.060460679] [joint_state_broadcaster]: Variance: 231659.275996 us

codecov · 2025-04-16T14:26:07Z

Codecov Report

Attention: Patch coverage is 97.24771% with 3 lines in your changes missing coverage. Please review.

Project coverage is 85.19%. Comparing base (0a20c8f) to head (ebc3290).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
..._state_broadcaster/src/joint_state_broadcaster.cpp	95.12%	0 Missing and 2 partials ⚠️
..._broadcaster/test/test_joint_state_broadcaster.cpp	98.48%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1640   +/-   ##
=======================================
  Coverage   85.19%   85.19%           
=======================================
  Files         126      127    +1     
  Lines       11916    12006   +90     
  Branches      997     1010   +13     
=======================================
+ Hits        10152    10229   +77     
- Misses       1452     1460    +8     
- Partials      312      317    +5

Flag	Coverage Δ
unittests	`85.19% <97.24%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...oint_state_broadcaster/joint_state_broadcaster.hpp	`100.00% <100.00%> (ø)`
..._broadcaster/test/test_joint_state_broadcaster.cpp	`96.92% <98.48%> (+0.31%)`	⬆️
..._state_broadcaster/src/joint_state_broadcaster.cpp	`85.79% <95.12%> (-3.65%)`	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jordan-palacios · 2025-04-16T14:37:01Z

Performance after latest commits.

[INFO] [1744720458.387674748] [joint_state_broadcaster]: Average update time: 104.401400 us
[INFO] [1744720458.387729533] [joint_state_broadcaster]: Variance: 666.211078 us
--
[INFO] [1744720459.388269041] [joint_state_broadcaster]: Average update time: 97.554600 us
[INFO] [1744720459.388316935] [joint_state_broadcaster]: Variance: 635.869019 us
--
[INFO] [1744720460.470225962] [joint_state_broadcaster]: Average update time: 105.671000 us
[INFO] [1744720460.470271104] [joint_state_broadcaster]: Variance: 130682.988159 us
--
[INFO] [1744720461.487041781] [joint_state_broadcaster]: Average update time: 99.057700 us
[INFO] [1744720461.487087239] [joint_state_broadcaster]: Variance: 649.182571 us
--
[INFO] [1744720462.514010777] [joint_state_broadcaster]: Average update time: 100.094300 us
[INFO] [1744720462.514054778] [joint_state_broadcaster]: Variance: 7139.876608 us
--
[INFO] [1744720463.565642664] [joint_state_broadcaster]: Average update time: 103.012900 us
[INFO] [1744720463.565686023] [joint_state_broadcaster]: Variance: 687.822734 us
--
[INFO] [1744720464.608627630] [joint_state_broadcaster]: Average update time: 101.937700 us
[INFO] [1744720464.608671023] [joint_state_broadcaster]: Variance: 676.558219 us
--
[INFO] [1744720465.684515309] [joint_state_broadcaster]: Average update time: 105.071300 us
[INFO] [1744720465.684570711] [joint_state_broadcaster]: Variance: 161106.798816 us
--
[INFO] [1744720466.708715393] [joint_state_broadcaster]: Average update time: 99.883600 us
[INFO] [1744720466.708758278] [joint_state_broadcaster]: Variance: 667.002251 us
--
[INFO] [1744720467.759692043] [joint_state_broadcaster]: Average update time: 102.654400 us
[INFO] [1744720467.759743639] [joint_state_broadcaster]: Variance: 668.791361 us

jordan-palacios · 2025-04-17T08:26:14Z

Performance after latest commits.

[INFO] [1744723251.403841025] [joint_state_broadcaster]: Average update time: 93.328200 us
[INFO] [1744723251.403888287] [joint_state_broadcaster]: Variance: 653.289085 us
--
[INFO] [1744723252.289001397] [joint_state_broadcaster]: Average update time: 85.812700 us
[INFO] [1744723252.289046780] [joint_state_broadcaster]: Variance: 545.987019 us
--
[INFO] [1744723253.209827955] [joint_state_broadcaster]: Average update time: 89.418700 us
[INFO] [1744723253.209875617] [joint_state_broadcaster]: Variance: 606.806190 us
--
[INFO] [1744723254.181901435] [joint_state_broadcaster]: Average update time: 94.719100 us
[INFO] [1744723254.181943388] [joint_state_broadcaster]: Variance: 7150.674195 us
--
[INFO] [1744723255.104042700] [joint_state_broadcaster]: Average update time: 89.827100 us
[INFO] [1744723255.104083336] [joint_state_broadcaster]: Variance: 571.054806 us
--
[INFO] [1744723256.046535039] [joint_state_broadcaster]: Average update time: 91.721900 us
[INFO] [1744723256.046578421] [joint_state_broadcaster]: Variance: 58434.660360 us
--
[INFO] [1744723256.981178385] [joint_state_broadcaster]: Average update time: 91.173800 us
[INFO] [1744723256.981223162] [joint_state_broadcaster]: Variance: 194645.084994 us
--
[INFO] [1744723257.930289307] [joint_state_broadcaster]: Average update time: 92.436200 us
[INFO] [1744723257.930340540] [joint_state_broadcaster]: Variance: 161048.665130 us
--
[INFO] [1744723258.845409847] [joint_state_broadcaster]: Average update time: 89.336100 us
[INFO] [1744723258.845449674] [joint_state_broadcaster]: Variance: 583.038737 us
--
[INFO] [1744723259.815155478] [joint_state_broadcaster]: Average update time: 94.531100 us
[INFO] [1744723259.815198062] [joint_state_broadcaster]: Variance: 161081.755233 us

- Includes a huge number (>550) of state interfaces - At the prints the average update time and variance

get_optional() is also more performant

bmagyar · 2025-04-17T10:21:52Z

Thank you for all the effort on this, the performance improvement is already superb but I'm also very happy about that massive reduction in variance.

jordan-palacios · 2025-04-17T12:55:06Z

Thank you for all the effort on this, the performance improvement is already superb but I'm also very happy about that massive reduction in variance.

My pleasure man. This is actually a nice break from the routine. :D

Performance after latest commits.

[INFO] [1744725296.145316901] [joint_state_broadcaster]: Average update time: 79.106100 us
[INFO] [1744725296.145362394] [joint_state_broadcaster]: Variance: 3.793643 us
--
[INFO] [1744725296.912327763] [joint_state_broadcaster]: Average update time: 74.606500 us
[INFO] [1744725296.912380387] [joint_state_broadcaster]: Variance: 2.370858 us
--
[INFO] [1744725297.710840156] [joint_state_broadcaster]: Average update time: 77.324300 us
[INFO] [1744725297.710892160] [joint_state_broadcaster]: Variance: 2.856730 us
--
[INFO] [1744725298.512992702] [joint_state_broadcaster]: Average update time: 77.753300 us
[INFO] [1744725298.513033306] [joint_state_broadcaster]: Variance: 2.283639 us
--
[INFO] [1744725299.320802269] [joint_state_broadcaster]: Average update time: 78.241700 us
[INFO] [1744725299.320844377] [joint_state_broadcaster]: Variance: 3.354681 us
--
[INFO] [1744725300.194178435] [joint_state_broadcaster]: Average update time: 84.872500 us
[INFO] [1744725300.194222263] [joint_state_broadcaster]: Variance: 160107.591044 us
--
[INFO] [1744725300.996058420] [joint_state_broadcaster]: Average update time: 77.732800 us
[INFO] [1744725300.996096360] [joint_state_broadcaster]: Variance: 2.465004 us
--
[INFO] [1744725301.825376099] [joint_state_broadcaster]: Average update time: 80.907000 us
[INFO] [1744725301.825426253] [joint_state_broadcaster]: Variance: 102490.028351 us
--
[INFO] [1744725302.643891628] [joint_state_broadcaster]: Average update time: 79.371900 us
[INFO] [1744725302.643944670] [joint_state_broadcaster]: Variance: 6432.001190 us
--
[INFO] [1744725303.452169592] [joint_state_broadcaster]: Average update time: 78.211600 us
[INFO] [1744725303.452222442] [joint_state_broadcaster]: Variance: 2.680825 us

jordan-palacios · 2025-04-17T13:39:54Z

Performance after latest commits.

This is the final fix.

[INFO] [1744726391.124390620] [joint_state_broadcaster]: Average update time: 25.697600 us
[INFO] [1744726391.124438080] [joint_state_broadcaster]: Variance: 3.110554 us
--
[INFO] [1744726391.389901107] [joint_state_broadcaster]: Average update time: 24.100800 us
[INFO] [1744726391.389943368] [joint_state_broadcaster]: Variance: 2.740039 us
--
[INFO] [1744726391.668664269] [joint_state_broadcaster]: Average update time: 25.334800 us
[INFO] [1744726391.668704795] [joint_state_broadcaster]: Variance: 1.270509 us
--
[INFO] [1744726391.946225775] [joint_state_broadcaster]: Average update time: 25.316400 us
[INFO] [1744726391.946264635] [joint_state_broadcaster]: Variance: 1.255291 us
--
[INFO] [1744726392.243318437] [joint_state_broadcaster]: Average update time: 27.165600 us
[INFO] [1744726392.243361865] [joint_state_broadcaster]: Variance: 40061.310577 us
--
[INFO] [1744726392.518379163] [joint_state_broadcaster]: Average update time: 25.009000 us
[INFO] [1744726392.518417231] [joint_state_broadcaster]: Variance: 2.103519 us
--
[INFO] [1744726392.784743900] [joint_state_broadcaster]: Average update time: 24.319800 us
[INFO] [1744726392.784782405] [joint_state_broadcaster]: Variance: 1.669928 us
--
[INFO] [1744726393.055926965] [joint_state_broadcaster]: Average update time: 25.000700 us
[INFO] [1744726393.055974185] [joint_state_broadcaster]: Variance: 2.187900 us
--
[INFO] [1744726393.322434122] [joint_state_broadcaster]: Average update time: 24.395100 us
[INFO] [1744726393.322479267] [joint_state_broadcaster]: Variance: 2.862796 us
--
[INFO] [1744726393.599603716] [joint_state_broadcaster]: Average update time: 25.296900 us
[INFO] [1744726393.599643378] [joint_state_broadcaster]: Variance: 1.293950 us

saikishor

The changes look really good to me. Thank you for the initiative :)

Just minor things

joint_state_broadcaster/src/joint_state_broadcaster.cpp

christophfroehlich

Looks great, and also tested in my setup

saikishor

Great job. Thank you so much!

LGTM

jordan-palacios force-pushed the fix/joint_state_broadcaster_performance branch 2 times, most recently from 1441c24 to 69cbe88 Compare April 16, 2025 08:17

jordan-palacios self-assigned this Apr 16, 2025

jordan-palacios added 4 commits April 17, 2025 10:27

Adding test that helps measure performance

6c4bf8e

- Includes a huge number (>550) of state interfaces - At the prints the average update time and variance

Use get_optional() instead of deprecated get_value()

7a65db2

get_optional() is also more performant

Auxiliary data for caching the interface to joint mapping

9a0a884

Auxiliary data for the joint state published data

262a229

jordan-palacios force-pushed the fix/joint_state_broadcaster_performance branch from 3ba6ec5 to 262a229 Compare April 17, 2025 08:28

Auxiliary data for the dynamic joint state published data

1d5e35a

For interface to joint mapping, cache the destination instead

aa0d0d8

jordan-palacios marked this pull request as ready for review April 17, 2025 13:40

github-actions bot requested review from anfemosa, arne48, christophfroehlich, destogl, fmauch, livanov93 and peterdavidfagan April 17, 2025 13:40

saikishor requested changes Apr 17, 2025

View reviewed changes

joint_state_broadcaster/src/joint_state_broadcaster.cpp Outdated Show resolved Hide resolved

joint_state_broadcaster/src/joint_state_broadcaster.cpp Outdated Show resolved Hide resolved

Removing debug log macro altogether

a7528e0

christophfroehlich previously approved these changes Apr 17, 2025

View reviewed changes

Try to get the latest value from the state interfaces only once

ebc3290

jordan-palacios dismissed christophfroehlich’s stale review via ebc3290 April 17, 2025 19:34

saikishor approved these changes Apr 17, 2025

View reviewed changes

saikishor requested a review from christophfroehlich April 17, 2025 20:36

christophfroehlich approved these changes Apr 17, 2025

View reviewed changes

saikishor merged commit 39c8ad3 into ros-controls:master Apr 18, 2025
23 of 25 checks passed

jordan-palacios deleted the fix/joint_state_broadcaster_performance branch April 18, 2025 13:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix joint_state_broadcaster performance issues #1640

Fix joint_state_broadcaster performance issues #1640

jordan-palacios commented Apr 16, 2025

jordan-palacios commented Apr 16, 2025

codecov bot commented Apr 16, 2025 •

edited

Loading

jordan-palacios commented Apr 16, 2025 •

edited

Loading

jordan-palacios commented Apr 17, 2025

bmagyar commented Apr 17, 2025

jordan-palacios commented Apr 17, 2025

jordan-palacios commented Apr 17, 2025

saikishor left a comment

christophfroehlich left a comment

saikishor left a comment

Fix joint_state_broadcaster performance issues #1640

Fix joint_state_broadcaster performance issues #1640

Conversation

jordan-palacios commented Apr 16, 2025

jordan-palacios commented Apr 16, 2025

codecov bot commented Apr 16, 2025 • edited Loading

Codecov Report

jordan-palacios commented Apr 16, 2025 • edited Loading

jordan-palacios commented Apr 17, 2025

bmagyar commented Apr 17, 2025

jordan-palacios commented Apr 17, 2025

jordan-palacios commented Apr 17, 2025

saikishor left a comment

Choose a reason for hiding this comment

christophfroehlich left a comment

Choose a reason for hiding this comment

saikishor left a comment

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2025 •

edited

Loading

jordan-palacios commented Apr 16, 2025 •

edited

Loading