Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CapMan visibility] surface CapMan summary in case of AllocationPolicyViolations #6052

Merged
merged 3 commits into from
Jun 25, 2024

Conversation

xurui-c
Copy link
Member

@xurui-c xurui-c commented Jun 23, 2024

https://getsentry.atlassian.net/browse/SNS-2798

Allocation policies are our mechanism for doing traffic management for Snuba queries. On Sentry issues webpage, the metadata regarding allocation policies are not surfaced.

Screenshot 2024-06-22 at 8 02 53 PM

In this PR, we relay the metadata to Sentry issues webpage so that they are displayed in that same section.

@xurui-c xurui-c marked this pull request as ready for review June 24, 2024 00:22
@xurui-c xurui-c requested a review from a team as a code owner June 24, 2024 00:22
@xurui-c xurui-c requested review from onewland and volokluev June 24, 2024 00:22
Copy link
Member

@evanh evanh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be work done here to have two concepts (QuotaAllowance and Summary) in one dictionary. Why not have them be separate attributes?

snuba/query/allocation_policies/__init__.py Outdated Show resolved Hide resolved
snuba/web/db_query.py Show resolved Hide resolved
@xurui-c
Copy link
Member Author

xurui-c commented Jun 24, 2024

I originally wanted to abstract away the fact that summary is an addition to quota allowances, since the intention is that any time quota allowances is retrieved, a summary should come with it. I updated pr to be your approach since I believe it's cleaner and easier to understand

@xurui-c xurui-c requested a review from evanh June 24, 2024 18:18
@@ -170,16 +170,25 @@ def quota_allowance(self) -> dict[str, dict[str, Any]]:
dict[str, dict[str, Any]], self.extra_data.get("quota_allowances", {})
)

@property
def summary(self) -> dict[str, Any]:
return cast(dict[str, Any], self.extra_data.get("summary"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where the summary key won't be in the extra_data? If so, I don't think this typing holds true. In that case the function will return None.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there will always be a summary

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added an assertion

@xurui-c xurui-c force-pushed the rachel/addSummaryToSentryIssues branch from 2e0f2cf to c7dbfd5 Compare June 24, 2024 19:14
@xurui-c xurui-c requested a review from evanh June 24, 2024 19:14
@xurui-c xurui-c force-pushed the rachel/addSummaryToSentryIssues branch from c7dbfd5 to 574a23a Compare June 24, 2024 22:21
@xurui-c xurui-c requested a review from volokluev June 25, 2024 17:52
@@ -418,7 +418,8 @@ def dataset_query(
{
"error": details,
"timing": timer.for_json(),
"quota_allowance": getattr(cause, "quota_allowance", {}),
"quota_allowance": getattr(cause, "quota_allowance", {})
| getattr(cause, "summary", {}),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evanh I initially had the two concepts in one dictionary because I was trying to avoid this. By previous design the summary field is a part of the quota_allowance dictionary, so if we kept them as separate attributes inside AllocationPolicyViolations, we have to re-combine them outside like this. I feel like this then becomes inconsistent with cases where the query is successful, and the quota_allowance dictionary contains the summary field

@xurui-c xurui-c force-pushed the rachel/addSummaryToSentryIssues branch from 6e59e53 to f491fa6 Compare June 25, 2024 20:37
@getsentry getsentry deleted a comment from xurui-c Jun 25, 2024
Copy link

codecov bot commented Jun 25, 2024

Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time.

❌ Failed Test Results:

Completed 2467 tests with 1 failed, 2462 passed and 4 skipped.

View the full list of failed tests

pytest

  • Test name: tests.web.test_db_query::test_db_query_success

    Traceback (most recent call last):
    File ".../tests/web/test_db_query.py", line 268, in test_db_query_success
    assert stats["quota_allowance"] == {
    AssertionError: assert {'details': {'BytesScannedRejectingPolicy': {'can_run': True,\n 'explanation': {'reason': 'within_limit '\n 'but '\n 'throttled',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': True,\n 'max_threads': 5,\n 'quota_unit': 'bytes',\n 'quota_used': 1560000000000,\n 'rejection_threshold': 2560000000000,\n 'suggestion': 'scan less bytes',\n 'throttle_threshold': 1280000000000},\n 'BytesScannedWindowAllocationPolicy': {'can_run': True,\n 'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'bytes',\n 'quota_used': 0,\n 'rejection_threshold': 1000000000000,\n 'suggestion': 'scan less '\n 'bytes',\n 'throttle_threshold': 10000000},\n 'ConcurrentRateLimitAllocationPolicy': {'can_run': True,\n 'explanation': {'overrides': {},\n 'reason': 'within '\n 'limit',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'concurrent_queries',\n 'quota_used': 1,\n 'rejection_threshold': 22,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 22},\n 'CrossOrgQueryAllocationPolicy': {'can_run': True,\n 'explanation': {'reason': 'pass_through',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'no_units',\n 'quota_used': 0,\n 'rejection_threshold': 1000000000000,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 1000000000000},\n 'ReferrerGuardRailPolicy': {'can_run': True,\n 'explanation': {'policy': 'referrer_guard_rail_policy',\n 'reason': 'within '\n 'limit',\n 'referrer': 'something',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'concurrent_queries',\n 'quota_used': 1,\n 'rejection_threshold': 100,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 50}},\n 'summary': {'rejected_by': {},\n 'threads_used': 5,\n 'throttled_by': {'policy': 'BytesScannedRejectingPolicy',\n 'quota_unit': 'bytes',\n 'quota_used': 1560000000000,\n 'suggestion': 'scan less bytes',\n 'throttle_threshold': 1280000000000}}} == {'BytesScannedRejectingPolicy': {'can_run': True,\n 'explanation': {'reason': 'within_limit but '\n 'throttled',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': True,\n 'max_threads': 5,\n 'quota_unit': 'bytes',\n 'quota_used': 1560000000000,\n 'rejection_threshold': 2560000000000,\n 'suggestion': 'scan less bytes',\n 'throttle_threshold': 1280000000000},\n 'BytesScannedWindowAllocationPolicy': {'can_run': True,\n 'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'bytes',\n 'quota_used': 0,\n 'rejection_threshold': 1000000000000,\n 'suggestion': 'scan less bytes',\n 'throttle_threshold': 10000000},\n 'ConcurrentRateLimitAllocationPolicy': {'can_run': True,\n 'explanation': {'overrides': {},\n 'reason': 'within '\n 'limit',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'concurrent_queries',\n 'quota_used': 1,\n 'rejection_threshold': 22,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 22},\n 'CrossOrgQueryAllocationPolicy': {'can_run': True,\n 'explanation': {'reason': 'pass_through',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'no_units',\n 'quota_used': 0,\n 'rejection_threshold': 1000000000000,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 1000000000000},\n 'ReferrerGuardRailPolicy': {'can_run': True,\n 'explanation': {'policy': 'referrer_guard_rail_policy',\n 'reason': 'within limit',\n 'referrer': 'something',\n 'storage_key': 'StorageKey.ERRORS_RO'},\n 'is_throttled': False,\n 'max_threads': 10,\n 'quota_unit': 'concurrent_queries',\n 'quota_used': 1,\n 'rejection_threshold': 100,\n 'suggestion': 'no_suggestion',\n 'throttle_threshold': 50},\n 'summary': {'rejected_by': {},\n 'threads_used': 5,\n 'throttled_by': {'policy': 'BytesScannedRejectingPolicy',\n 'quota_unit': 'bytes',\n 'quota_used': 1560000000000,\n 'suggestion': 'scan less bytes',\n 'throttle_threshold': 1280000000000}}}
    Common items:
    {'summary': {'rejected_by': {},
    'threads_used': 5,
    'throttled_by': {'policy': 'BytesScannedRejectingPolicy',
    'quota_unit': 'bytes',
    'quota_used': 1560000000000,
    'suggestion': 'scan less bytes',
    'throttle_threshold': 1280000000000}}}
    Left contains 1 more item:
    {'details': {'BytesScannedRejectingPolicy': {'can_run': True,
    'explanation': {'reason': 'within_limit '
    'but '
    'throttled',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': True,
    'max_threads': 5,
    'quota_unit': 'bytes',
    'quota_used': 1560000000000,
    'rejection_threshold': 2560000000000,
    'suggestion': 'scan less bytes',
    'throttle_threshold': 1280000000000},
    'BytesScannedWindowAllocationPolicy': {'can_run': True,
    'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'bytes',
    'quota_used': 0,
    'rejection_threshold': 1000000000000,
    'suggestion': 'scan less '
    'bytes',
    'throttle_threshold': 10000000},
    'ConcurrentRateLimitAllocationPolicy': {'can_run': True,
    'explanation': {'overrides': {},
    'reason': 'within '
    'limit',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'concurrent_queries',
    'quota_used': 1,
    'rejection_threshold': 22,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 22},
    'CrossOrgQueryAllocationPolicy': {'can_run': True,
    'explanation': {'reason': 'pass_through',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'no_units',
    'quota_used': 0,
    'rejection_threshold': 1000000000000,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 1000000000000},
    'ReferrerGuardRailPolicy': {'can_run': True,
    'explanation': {'policy': 'referrer_guard_rail_policy',
    'reason': 'within '
    'limit',
    'referrer': 'something',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'concurrent_queries',
    'quota_used': 1,
    'rejection_threshold': 100,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 50}}}
    Right contains 5 more items:
    {'BytesScannedRejectingPolicy': {'can_run': True,
    'explanation': {'reason': 'within_limit but '
    'throttled',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': True,
    'max_threads': 5,
    'quota_unit': 'bytes',
    'quota_used': 1560000000000,
    'rejection_threshold': 2560000000000,
    'suggestion': 'scan less bytes',
    'throttle_threshold': 1280000000000},
    'BytesScannedWindowAllocationPolicy': {'can_run': True,
    'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'bytes',
    'quota_used': 0,
    'rejection_threshold': 1000000000000,
    'suggestion': 'scan less bytes',
    'throttle_threshold': 10000000},
    'ConcurrentRateLimitAllocationPolicy': {'can_run': True,
    'explanation': {'overrides': {},
    'reason': 'within '
    'limit',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'concurrent_queries',
    'quota_used': 1,
    'rejection_threshold': 22,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 22},
    'CrossOrgQueryAllocationPolicy': {'can_run': True,
    'explanation': {'reason': 'pass_through',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'no_units',
    'quota_used': 0,
    'rejection_threshold': 1000000000000,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 1000000000000},
    'ReferrerGuardRailPolicy': {'can_run': True,
    'explanation': {'policy': 'referrer_guard_rail_policy',
    'reason': 'within limit',
    'referrer': 'something',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'concurrent_queries',
    'quota_used': 1,
    'rejection_threshold': 100,
    'suggestion': 'no_suggestion',
    'throttle_threshold': 50}}
    Full diff:
    {
    - 'BytesScannedRejectingPolicy': {'can_run': True,
    + 'details': {'BytesScannedRejectingPolicy': {'can_run': True,
    & ++++++++++++
    - 'explanation': {'reason': 'within_limit but '
    & ----
    + 'explanation': {'reason': 'within_limit '
    & ++++++++++++
    + 'but '
    - 'throttled',
    + 'throttled',
    & ++++++++++++
    - 'storage_key': 'StorageKey.ERRORS_RO'},
    + 'storage_key': 'StorageKey.ERRORS_RO'},
    & ++++++++++++
    - 'is_throttled': True,
    - 'max_threads': 5,
    - 'quota_unit': 'bytes',
    - 'quota_used': 1560000000000,
    - 'rejection_threshold': 2560000000000,
    - 'suggestion': 'scan less bytes',
    - 'throttle_threshold': 1280000000000},
    - 'BytesScannedWindowAllocationPolicy': {'can_run': True,
    - 'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},
    - 'is_throttled': False,
    & ^^^^
    + 'is_throttled': True,
    & +++++ ^^^
    - 'max_threads': 10,
    & ^^
    + 'max_threads': 5,
    & +++++ ^
    - 'quota_unit': 'bytes',
    + 'quota_unit': 'bytes',
    & +++++
    - 'quota_used': 0,
    + 'quota_used': 1560000000000,
    & +++++ ++++++++++++
    - 'rejection_threshold': 1000000000000,
    & ^^^
    + 'rejection_threshold': 2560000000000,
    & +++++ ^^^
    - 'suggestion': 'scan less bytes',
    + 'suggestion': 'scan less bytes',
    & +++++
    - 'throttle_threshold': 10000000},
    + 'throttle_threshold': 1280000000000},
    & +++++ +++++
    + 'BytesScannedWindowAllocationPolicy': {'can_run': True,
    + 'explanation': {'storage_key': 'StorageKey.ERRORS_RO'},
    + 'is_throttled': False,
    + 'max_threads': 10,
    + 'quota_unit': 'bytes',
    + 'quota_used': 0,
    + 'rejection_threshold': 1000000000000,
    + 'suggestion': 'scan less '
    + 'bytes',
    + 'throttle_threshold': 10000000},
    - 'ConcurrentRateLimitAllocationPolicy': {'can_run': True,
    + 'ConcurrentRateLimitAllocationPolicy': {'can_run': True,
    & ++++++++++++
    - 'explanation': {'overrides': {},
    + 'explanation': {'overrides': {},
    & ++++++++++++
    + 'reason': 'within '
    + 'limit',
    + 'storage_key': 'StorageKey.ERRORS_RO'},
    + 'is_throttled': False,
    + 'max_threads': 10,
    + 'quota_unit': 'concurrent_queries',
    + 'quota_used': 1,
    + 'rejection_threshold': 22,
    + 'suggestion': 'no_suggestion',
    + 'throttle_threshold': 22},
    + 'CrossOrgQueryAllocationPolicy': {'can_run': True,
    + 'explanation': {'reason': 'pass_through',
    + 'storage_key': 'StorageKey.ERRORS_RO'},
    + 'is_throttled': False,
    + 'max_threads': 10,
    + 'quota_unit': 'no_units',
    + 'quota_used': 0,
    + 'rejection_threshold': 1000000000000,
    + 'suggestion': 'no_suggestion',
    + 'throttle_threshold': 1000000000000},
    + 'ReferrerGuardRailPolicy': {'can_run': True,
    + 'explanation': {'policy': 'referrer_guard_rail_policy',
    'reason': 'within '
    'limit',
    + 'referrer': 'something',
    'storage_key': 'StorageKey.ERRORS_RO'},
    'is_throttled': False,
    'max_threads': 10,
    'quota_unit': 'concurrent_queries',
    'quota_used': 1,
    - 'rejection_threshold': 22,
    & ^^
    + 'rejection_threshold': 100,
    & ^^^
    'suggestion': 'no_suggestion',
    - 'throttle_threshold': 22},
    & ^^
    + 'throttle_threshold': 50}},
    & ^^^
    - 'CrossOrgQueryAllocationPolicy': {'can_run': True,
    - 'explanation': {'reason': 'pass_through',
    - 'storage_key': 'StorageKey.ERRORS_RO'},
    - 'is_throttled': False,
    - 'max_threads': 10,
    - 'quota_unit': 'no_units',
    - 'quota_used': 0,
    - 'rejection_threshold': 1000000000000,
    - 'suggestion': 'no_suggestion',
    - 'throttle_threshold': 1000000000000},
    - 'ReferrerGuardRailPolicy': {'can_run': True,
    - 'explanation': {'policy': 'referrer_guard_rail_policy',
    - 'reason': 'within limit',
    - 'referrer': 'something',
    - 'storage_key': 'StorageKey.ERRORS_RO'},
    - 'is_throttled': False,
    - 'max_threads': 10,
    - 'quota_unit': 'concurrent_queries',
    - 'quota_used': 1,
    - 'rejection_threshold': 100,
    - 'suggestion': 'no_suggestion',
    - 'throttle_threshold': 50},
    'summary': {'rejected_by': {},
    'threads_used': 5,
    'throttled_by': {'policy': 'BytesScannedRejectingPolicy',
    'quota_unit': 'bytes',
    'quota_used': 1560000000000,
    'suggestion': 'scan less bytes',
    'throttle_threshold': 1280000000000}},
    }

Copy link
Member

@volokluev volokluev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some small changes but otherwise looks good

@@ -158,28 +158,33 @@ class AllocationPolicyViolations(SerializableException):
"""

def __str__(self) -> str:
return f"{self.message}, details: {self.violations}"
return f"{self.message}, info: {self.violations | self.summary}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return f"{self.message}, info: {self.violations | self.summary}"
return f"{self.message}, info: {"violations" self.violations, "summary": self.summary}"

Comment on lines 176 to 178
summary = self.quota_allowance.get("summary")
assert summary is not None
return {"summary": summary}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
summary = self.quota_allowance.get("summary")
assert summary is not None
return {"summary": summary}
return self.quota_allowance.get("summary", {})

@xurui-c xurui-c force-pushed the rachel/addSummaryToSentryIssues branch from f491fa6 to 72675b4 Compare June 25, 2024 21:29
@xurui-c xurui-c merged commit 8530e2e into master Jun 25, 2024
28 checks passed
@xurui-c xurui-c deleted the rachel/addSummaryToSentryIssues branch June 25, 2024 22:07
@xurui-c xurui-c changed the title surface CapMan summary in case of AllocationPolicyViolations [CapMan visibility] surface CapMan summary in case of AllocationPolicyViolations Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants