fix[tests]: increase overlap threshold in test_mode_solver_data_sort #2229
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This test has been failing sporadically. I was able to reproduce the issue locally when testing just this function instead of the whole test_monitor_data.py file. Changing the rng seed or state would then randomly cause the test to start working again. I traced this to the
unsorting
array that was getting generated (line 542 in the code change). When the random permutation was [3, 1, 2, 0] (i.e. - the first and last modes were swapped), they were not getting re-sorted properly. I was able to reproduce reliably by just hardcoding the permutation to this for all the frequencies.It seems like the default
overlap_thresh
argument to theoverlap_sort
function is not high enough to properly sort these modes. Here, I've specified a higheroverlap_thresh
value in order for the test to pass assuming that the threshold being too low is mostly related to the test operating on generated data instead of actual mode solver runs. However, I'm wondering if this is an indication that the default should be made higher instead?