Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(eap): Ensure reliabilities are returned more consistently #6715

Merged

Conversation

Zylphrex
Copy link
Member

@Zylphrex Zylphrex commented Jan 2, 2025

When extrapolating, if there's no data, we skip returning reliabilities. This is problematic as we want to line up each reliability value with the corresponding result so those arrays must be the same length. This fills it with RELIABILITY_UNSPECIFIED.

When extrapolating, if there's no data, we skip returning reliabilities. This is
problematic as we want to line up each reliability value with the corresponding
result so those arrays must be the same length. This fills it with `RELIABILITY_UNSPECIFIED`.
Comment on lines +177 to +179
# This isn't quite right but all non extrapolated aggregates
# are assumed to be present.
not extrapolation_context.is_extrapolated
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to change too much existing behaviour in this PR but for whatever reason, when using a non aggregate in a timeseries query, we always consider as data_present=True.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this come up in discuss-sns when a user was confused about it. Seems like it might be a bug.

Copy link
Member Author

@Zylphrex Zylphrex Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I didn't want to change too many behaviours in this one PR so I kept this behaviour for now but it does feel like a bug.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kylemumma can you look into this and file a ticket on eap-planning to address this? I am not following what the bug is really but if either of you have an idea of the issue, we should track it and even fix it. Let's not leave this unaddressed.

Copy link

codecov bot commented Jan 2, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
2784 1 2783 5
View the top 1 failed tests by shortest run time
tests.web.rpc.v1.test_endpoint_trace_item_table.test_endpoint_trace_item_table_extrapolation.TestTraceItemTableWithExtrapolation::test_count_reliability_with_group_by
Stack Traces | 0.31s run time
Traceback (most recent call last):
  File ".../v1/test_endpoint_trace_item_table/test_endpoint_trace_item_table_extrapolation.py", line 378, in test_count_reliability_with_group_by
    response = EndpointTraceItemTable().execute(message)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../web/rpc/__init__.py", line 83, in execute
    return self.__after_execute(in_msg, out, error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../web/rpc/__init__.py", line 104, in __after_execute
    raise error
  File ".../web/rpc/__init__.py", line 79, in execute
    out = self._execute(in_msg)
          ^^^^^^^^^^^^^^^^^^^^^
  File ".../rpc/v1/endpoint_trace_item_table.py", line 320, in _execute
    column_values = _convert_results(in_msg, res.result.get("data", []))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../rpc/v1/endpoint_trace_item_table.py", line 215, in _convert_results
    extrapolation_context.reliability
  File ".../local/lib/python3.11/functools.py", line 1001, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File ".../rpc/common/aggregation.py", line 189, in reliability
    lower_bound, upper_bound = _calculate_approximate_ci_percentile_levels(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../rpc/common/aggregation.py", line 607, in _calculate_approximate_ci_percentile_levels
    return (lower_index / n, upper_index / n)
            ~~~~~~~~~~~~^~~
ZeroDivisionError: float division by zero

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

@Zylphrex Zylphrex marked this pull request as ready for review January 3, 2025 13:54
@Zylphrex Zylphrex requested review from a team as code owners January 3, 2025 13:54
Copy link
Member

@kylemumma kylemumma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked your use of comments and your code was easy to read

Comment on lines +177 to +179
# This isn't quite right but all non extrapolated aggregates
# are assumed to be present.
not extrapolation_context.is_extrapolated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this come up in discuss-sns when a user was confused about it. Seems like it might be a bug.

@Zylphrex Zylphrex merged commit b22e0cc into master Jan 3, 2025
31 checks passed
@Zylphrex Zylphrex deleted the txiao/fix/ensure-reliabilities-are-returned-more-consistently branch January 3, 2025 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants