diff --git a/src/content/docs/scorecards/limitations.mdx b/src/content/docs/scorecards/limitations.mdx new file mode 100644 index 00000000000..8b81734c0c5 --- /dev/null +++ b/src/content/docs/scorecards/limitations.mdx @@ -0,0 +1,13 @@ +--- +title: Limitations +--- +As Scorecards are in a public preview, the following restrictions apply: + +- **Entity limit:** Maximum of 5,000 entities per rule. +- **Rule limit:** Maximum of 1,000 rules per organization. +- **Allowed dimensions:** Only `team`, `department`, and `environment` are allowed. +- **Scorecard association:** If a rule is associated with more than one scorecard, only the last scorecard will consider the rule. + These limitations ensure system performance and manageability while in limited preview mode. + + + diff --git a/src/content/docs/scorecards/rule-view.mdx b/src/content/docs/scorecards/rule-view.mdx new file mode 100644 index 00000000000..c4fcd22d0bd --- /dev/null +++ b/src/content/docs/scorecards/rule-view.mdx @@ -0,0 +1,22 @@ +--- +title: Rule view +--- + Rules view + +Rules are the building blocks of Scorecards. Rules contain specific performance conditions for compliance. The rule view enables users to assess and manage individual rules, offering visibility into entities to meet or fail the conditions of each rule. + +- **Rule description:** Each rule includes a description, New Relic Query Language (NRQL) statement, and scope (the entities or accounts it evaluates). Users can examine these rules to understand the purpose and criteria of each rule. + +- **Evaluation summary:** + + - **Status (Pass/Fail):** It shows a list of all the entities evaluated by the rule, showing whether each one passes or fails. This allows users to quickly identify problem areas. + + - **Rule score calculation:** Each rule’s score is calculated as a percentage of passing checks to total checks, enabling quick assessment of rule adherence. + +- **Manage entities:** Users can update the scope of each rule by adding or removing specific entities. + +- **Enable/disable rules:** It allows users to toggle rules on or off, based on relevance to current engineering standards. diff --git a/src/content/docs/scorecards/scorecards-api.mdx b/src/content/docs/scorecards/scorecards-api.mdx new file mode 100644 index 00000000000..25c1a10b6d2 --- /dev/null +++ b/src/content/docs/scorecards/scorecards-api.mdx @@ -0,0 +1,401 @@ +--- +title: Scorecards API +--- +All rules of Scorecards belong to the rules collection of the Scorecard. Therefore, both `scorecardId` and `collectionId` are needed to know which scorecard owns a rule. + +Below is an example of creating a rule that checks whether your APM services have alerts defined. + +1. Create a scorecard + +```sql +mutation CreateScorecard($description: String, $name: String!, $id: ID!) { + entityManagementCreateScorecard( + scorecardEntity: {description: $description, name: $name, scope: {id: $id, type: ACCOUNT}} + ) { + entity { + description + id + rules { + id // COLLECTION ID + } + name + } + } +} +// PARAMETERS +{ + "description": "NR Engineering Best Practices", + "name": "Measure Your Engineering Best Practices", + "id": 1 +} +``` + +2. Create a rule + +```sql +mutation CreateRule($name: String!, $description: String, $query: String!, $accounts: [Int!]!) { + entityManagementCreateScorecardRule( + scorecardRuleEntity: { + name: $name, + description: $description + enabled: true, + nrqlEngine: { + accounts: $accounts, + query: $query + }, + scope: {id:1 , type: ACCOUNT}} + ) { + entity { + id // RULE Id + } + } +} + + +// PARAMETERS +{ + "name": "APM Services Have Alerts Defined", + "description": "Check that APM services have alerts associated with them", + "query": "SELECT if(latest(alertSeverity) != 'NOT_CONFIGURED', 1, 0) as 'score' FROM Entity WHERE type = 'APM-APPLICATION' AND tags.nr.team IS NOT NULL AND tags.environment IS NOT NULL FACET id as 'entityGuid', tags.nr.team as 'team', tags.environment as 'environment' LIMIT MAX SINCE 1 day ago", + "accounts": [1] +} + + +// ADD RULE TO SCORECARD COLLECTION +mutation AddRuleToCollection($collectionId: ID!, $rules: [ID!]!){ + entityManagementAddCollectionMembers( + collectionId: $collectionId + ids: $rules + ) +} +// PARAMETERS +{ + "collectionId": "", // Collection ID is from the rule.id from scorecard enntity + "rules": [] // Provide list of all rule ids which are generated during rule creation. +} +``` + +### Use cases + + 1. **Get rules in a Scorecard:** Retrieve all rules associated with a specific scorecard. + + **GraphQL Query:** + +```sql + query FetchScorecardDetails ($scorecardId: ID!) { + actor { + entityManagement { + entity(id: $scorecardId) { + ... on EntityManagementScorecardEntity { + name + description + rules { + id + } + } + } + } + } + } +``` + +**Fields:** + +- `scorecardId:` The id of the scorecard. + +**Response:** + +- `rules.id` is the next query `collectionId`. + +```sql + query FetchRulesCollection($rulesId: ID!) { + actor { + entityManagement { + collectionElements(filter: {collectionId: {eq: $rulesId}}) { + items { + ... on EntityManagementScorecardRuleEntity { + id + name + nrqlEngine { + accounts + query + } + } + } + nextCursor + } + } + } +} +``` + +**Fields:** + +- `collectionId` is the value from previous response `rules.id`. + +2. **Get the scorecard of a rule** retrieve the scorecard associated with a specific rule. + + **GraphQL Query:** + + First, find the collections that contain the rule. + +```sql + query FindRuleOwnerCollections($ruleId: ID!) { + actor { + entityManagement { + relationships(filter: {targetId: {eq: $ruleId}, type: {eq: "HAS_MEMBER"}}) { + items { + source { + id + type + } + type + } + } + } + } +} +``` + +Then, you need to find the parents of the collection. + +```sql +query FetchCollectionParent($collectionId: ID!) { + actor { + entityManagement { + relationships( + filter: {targetId: {eq: $collectionId}, type: {eq: "HAS_COLLECTION"}} + ) { + items { + source { + id + type + } + target { + id + type + } + } + } + } + } +} +``` + + **Fields:** + + - `ruleId?` + - + + 3. **Create a scorecard:** Create a new scorecard. + + **GraphQL Mutation:** + +```sql + mutation CreateScorecard($name: String!, $desc: String, $accountId: ID!) { + entityManagementCreateScorecard( + scorecardEntity: {description: $desc, name: $name, scope: {id: $accountId, type: ACCOUNT}} + ) { + entity { + id + rules { + id + } + } + } +} +``` + + **Fields:** + + - `name:` The name of the scorecard. + - `description:` A brief description of the scorecard. + - `accountId:` Account where the entity will be stored. + + **Response:** + + - `rules.id` Contains the id you need to associate rules to a scorecard. + +4. **Create a Rule:** Add a new rule to a scorecard. + + **GraphQL Mutation:** + +```sql +mutation CreateRule($name: String!, $description: String, $query: String!, $queryAccounts: [Int!]!, $account: ID!) { + entityManagementCreateScorecardRule( + scorecardRuleEntity: { + name: $name, + description: $description + enabled: true, + nrqlEngine: { + accounts: $queryAccounts, + query: $query + }, + tags: { + key: "demo", values: "true" + }, + scope: {id:$account , type: ACCOUNT}} + ) { + entity { + + id + + } + } +} +``` + +**Fields:** + +- `name:` The name of the scorecard. +- `description:` A brief description of the scorecard. +- `accountId:` Account where the entity will be stored. +- `query:` A nrql query to read data. + - A `score` is a mandatory selection. + - A FACET with at least and entity id (entity.guid, entity.id or entityGuid) is mandatory. +- `queryAccounts:` A list of accounts where the rule should execute the query. + + Create a scorecard, follow step no: 3 and all rule guid to scorecard collection id. + +```sql +mutation addRuleToCollection($ruleId: ID!, $collectionId: ID!) { + entityManagementAddCollectionMembers( + collectionId: $collectionId + ids: [$ruleId] + ) +} +``` + +- Collection id is the value from previous response `rules.id` on create scorecard response. + +5. **Delete a Scorecard or Rule:** Remove an existing scorecard. + + **GraphQL Mutation:** + +```sql +mutation DeleteEntity ($id: ID!) { + entityManagementDelete( + id: $id + ) { + id + } +} +``` + +**Fields:** + +- `id:` The target scorecard/rule id. + +6. **Modify a Scorecard** Update the details of an existing scorecard. + + **GraphQL Mutation:** + + The update operation is similar to creating one, but the mutation name is different. + +```sql +mutation updateScorecard($id: ID!, $description: String, $name: String!) { + entityManagementUpdateScorecard( + id: $id + scorecardEntity: {description: $description, name: $name} + ) { + entity { + name + id + rules { + id + } + } + } +} +``` + +**Fields:** + +- `id:` Scorecard entity guid. +- `description:` Scorecard description. +- `name:` Scorecard name. + +7. **Modify a Rule:** Update the details of an existing rule. + + **GraphQL Mutation:** + + Fields are the same for update and create, with different name in the mutation and with id of entity. + +```sql +mutation UpdateRule($ruleId: ID!, $name: String!, $description: String, $query: String!, $queryAccounts: [Int!]!, $enabled: Boolean) { + entityManagementUpdateScorecardRule( + id: $ruleId + scorecardRuleEntity: {description: $description, name: $name, enabled: $enabled, nrqlEngine: {accounts: $queryAccounts, query: $query}} + ) { + entity { + id + name + description + nrqlEngine { + accounts + query + } + } + } +} +``` + +**Fields:** + +- `ruleId:` The unique identifier of the rule. +- `name:` Name of the scorecard rule. +- `description:` Description of the scorecard rule. +- `query:` NRQL query to read data. +- `queryAccounts:` List of accounts where the data need to be read. +- `enabled:` Is rule enabled? + +8. **Modify the Query:** Change the query used by a specific rule. + + **GraphQL Mutation:** + +```sql +mutation UpdateRule($id: ID!, $query: String!, $queryAccounts: [Int!]!, $account: ID!) { + entityManagementUpdateScorecardRule( + id: $id + scorecardRuleEntity: { nrqlEngine: {accounts: $queryAccounts, query: $Query}} + ) { + entity { + id + nrqlEngine { + accounts + query + } + } + } +} +``` + +**Fields:** + +- `id`: Entity Id of the rule. +- `query`: New query. +- `Accounts`: Target list of accounts IDs, if empty, the query won’t run in any account. + + 9. **Activate or Deactivate the Rule** Enable or disable a rule within a scorecard. + + **GraphQL Mutation:** + +```sql +mutation UpdateRule($ruleId: ID!,$enabled: boolean) { + entityManagementUpdateScorecardRule( + id: $ruleId + scorecardRuleEntity: {enabled: $enabled} + ) { + entity { + id + name + } + } +} +``` + +**Fields:** + + - `ruleId:` The unique identifier of the rule. + - `enabled:` Is rule enabled? + diff --git a/src/content/docs/scorecards/scorecards-rules.mdx b/src/content/docs/scorecards/scorecards-rules.mdx new file mode 100644 index 00000000000..b4140b4da3a --- /dev/null +++ b/src/content/docs/scorecards/scorecards-rules.mdx @@ -0,0 +1,117 @@ +--- +title: Scorecards rules +--- +To effectively understand scorecards, it is essential to recognize that rules are predefined criteria or conditions used to evaluate performance against specific objectives. These rules form the foundation for assessing whether performance metrics meet the desired standards. + +Each rule typically comprises one or more checks, which are specific queries or tests that determine if a condition is met, often resulting in a binary outcome (true or false). Rules are aligned with strategic goals, ensuring that the scorecard reflects the organization's priorities. + +By providing structured criteria for evaluation, rules enable consistent and objective performance assessments, offering actionable insights and supporting continuous improvement within the organization. + +### Score evaluation + +Score evaluation is a systematic process used to assess and quantify performance across various metrics and criteria. Some evaluation parameters are listed below: + +- **Check score:** + A check is a specific query executed against an entity to determine if a condition is met. The outcome of a check can only be either true or false. Each check is associated with a single rule and represents the smallest evaluative unit within the rule. + +- **Rule score:** + The rule score reflects the performance of all checks associated with that rule. + +- **Scorecard score:** + The scorecard score represents the average performance across all the rules contained within the scorecard. + +- **Summary score:** + The overall score represents the average performance across all the scorecards. + + +### What is NRQL? +Refer the [NRQL](/docs/nrql/get-started/introduction-nrql-new-relics-query-language/) section for more information. + +### Some quick references for Scorecards rules + + + + + Rules are based on NRQL (New Relic Query Language) queries executed against the New Relic Database (NRDB). Each rule spans multiple entities and checks whether each entity meets specific conditions by evaluating to either `true` or `false`. Essentially, a rule verifies whether each entity complies with the defined criteria. + + + + + A Rule NRQL must contain the following components: + + 1. **Facet by** `entityGuid:` This identifies each entity individually within the query. + + 1. **Score Column:** The score column reflects the rule’s result for each entity. It evaluates as: `1:` The entity meets the rule (passes). `0:` The entity does not meet the rule (fails). `-1:` No data is available for the entity (defaults to failure). + +Example of NRQL rule: + +The following example checks if each APM service has alerts configured: + +```sql +FROM Entity +SELECT if(latest(alertSeverity) != 'NOT_CONFIGURED', 1, 0) as 'score' +WHERE type = 'APM-APPLICATION' +FACET id as 'entityGuid' +LIMIT MAX +SINCE 1 day ago +``` + +**Tip:** You can filter entities by specifying attributes, like `type`, within the `Entity` table. + + + + + The `Entity` event in NRDB is where you’ll find all attributes related to your entities, allowing you to explore and create rules. For example, to explore the `Entity` event data, run: + +```sql + +SELECT * FROM Entity LIMIT 10 SINCE 1 day ago + +``` + +### Example use cases for Entity Event + +- **Check for alerts configuration:** The example query above checks if APM services have alerts configured. +- ** Check for tags:** To confirm that entities are tagged (for example, assigned to specific teams), you could use: + +```sql + +SELECT id as EntityGuid, if(tags.nr.team IS NOT NULL, 1, 0) as 'score' +FROM Entity +SINCE 1 day ago + +``` + + + + +You can add the following dimensions to your rules to make your data more granular: + +- Environment +- Team +- Department + +To include these dimensions, add them to the `FACET` statement in your NRQL query. Here’s an example: + +```sql + +SELECT if(latest(alertSeverity) != 'NOT_CONFIGURED', 1, 0) as 'score' +FROM Entity +WHERE type = 'APM-APPLICATION'  +  AND tags.nr.team IS NOT NULL  +  AND tags.environment IS NOT NULL +FACET id as 'entityGuid', tags.nr.team as 'team', tags.environment as 'environment' +LIMIT MAX +SINCE 1 day ago +``` + + + diff --git a/src/content/docs/scorecards/scorecards.mdx b/src/content/docs/scorecards/scorecards.mdx new file mode 100644 index 00000000000..4ea99e4a11b --- /dev/null +++ b/src/content/docs/scorecards/scorecards.mdx @@ -0,0 +1,29 @@ +--- +title: Scorecards overview +--- + + We're still working on this feature, but we'd love for you to try it out! + + This feature is currently provided as part of a preview program pursuant to our [pre-release policies](/docs/licenses/license-information/referenced-policies/new-relic-pre-release-policy). + + +Scorecards page offers a centralized, high-level view of all Scorecards. It provides insights into the organization's adherence to engineering standards and overall performance. This view enables leaders and teams to monitor performance metrics across multiple Scorecards and identify areas requiring immediate action. + +By using this approach, organizations can monitor and improve performance across various predefined areas, ensuring that short-term actions align with long-term objectives. Scorecards transform strategic goals into measurable outcomes, driving continuous improvement and better decision-making. + +### Get started with Scorecards + + Scorecards overview + +### Access Scorecards + + Go to **[one.newrelic.com](https://one.newrelic.com) > All Capabilities >** and click **Scorecards** + + +The above shown dashboard area is dynamic in nature. This dynamic quality is essential for providing real-time insights and enabling users to interact with the data in meaningful ways. This dynamicity serves as the starting point for users to access and manage all scorecards, offering a comprehensive view of engineering performance across the organization. + +Use the [filters](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/search-filter-entities/#find) to customize the view based on your preferences. You can filter by department, team, scorecard name, or by scorecard status. The dashboard provides a high-level overview of the organization's performance, with the ability to drill down into specific scorecards for detailed insights. You can see this type of capability into [Single Scorecards view](/docs/scorecards/single-scorecard-view/). \ No newline at end of file diff --git a/src/content/docs/scorecards/send-data-to-nr-to-build-rules.mdx b/src/content/docs/scorecards/send-data-to-nr-to-build-rules.mdx new file mode 100644 index 00000000000..231d0c38d5c --- /dev/null +++ b/src/content/docs/scorecards/send-data-to-nr-to-build-rules.mdx @@ -0,0 +1,8 @@ +--- +title: Send data To NR to build rules +--- +Scorecards are driven by data—the more data you integrate, the broader the range of use cases you can address. Since Scorecards rules are based on NRQL queries, the data stored in NRDB forms the foundation of all rule evaluations. By sending diverse data to New Relic, you can build custom rules that monitor exactly what matters most for your organization. + +Explore these 20 methods for sending data to New Relic to help you design comprehensive rules and maximize your Scorecards' potential: + +[click here](https://newrelic.com/blog/how-to-relic/20-ways-send-data-new-relic) \ No newline at end of file diff --git a/src/content/docs/scorecards/single-scorecard-view.mdx b/src/content/docs/scorecards/single-scorecard-view.mdx new file mode 100644 index 00000000000..ad9123f9a2f --- /dev/null +++ b/src/content/docs/scorecards/single-scorecard-view.mdx @@ -0,0 +1,24 @@ +--- +title: Single Scorecard view +--- + Single Scorecard view + +The single Scorecard view provides a deep dive into the specific metrics and rules associated with a single scorecard, including historical performance data. This view is crucial for understanding detailed scoring criteria and driving targeted actions to improve team performance. + +Clicking on the name of Scorecards takes you to deep dive into the the details of the selected Scorecards at any level. In the above screen, clicking on the selected Scorecards **NR Obserability best practices** will take you to the granular details as mentioned in the below Scorecards performance parameters section. + +- **Scorecards performance parameters:** + + - **Historical data:** The historical data provides visibility about trends in Scorecards performance over time, with data visualisation options to track improvements or declines. + - **Current score:** The current score is the overall score calculated as the average of all rule scores within the Scorecards. It represents an immediate sense of compliance with engineering standards. + - **Rules and scoring breakdown:** Each rule’s score is displayed alongside its condition, scope, and evaluation status. Users can drill down to see the specific checks each rule encompasses and review pass/fail statuses. + +- **Grouping options:** + + - **Group By - None:** This option allows switching from the **Team** view to none for viewing the data. + - **Group By - Team:** This option enables the grouping of information retrieved on the dashboard into the **Team** view. It shows the grouping of information of a **Team** for the selected single scorecard. + diff --git a/src/content/docs/scorecards/teams-view.mdx b/src/content/docs/scorecards/teams-view.mdx new file mode 100644 index 00000000000..9dbf193a379 --- /dev/null +++ b/src/content/docs/scorecards/teams-view.mdx @@ -0,0 +1,18 @@ +--- +title: Teams view +--- + Teams view + +The Teams view allows users to visualize Scorecards performance organized by teams. This view aggregates Scorecards data at the team level, making it easier to monitor, compare, and assess how individual teams are meeting organizational engineering standards. + +This view is particularly useful for managers and leaders who need a team-centric perspective of Scorecards compliance and performance. It allows you to access the Teams View by clicking on the **Group By** button on the top right corner of the screen. If you are not enrolled in the teams product, please contact your account manager to be enrolled in Teams product public preview. + +- **Team grouped score display:** + +- Each team’s score is calculated as the average of the Scorecards scores associated with that team, providing an at-a-glance view of each team’s adherence to standards. + +- Team status indicators: Each team’s performance status is categorized by color or label such as, **Good,** **Below Goal,** **Need Attention,** to signal areas that require immediate attention. diff --git a/src/nav/root.yml b/src/nav/root.yml index e6031683839..6104f1a05bb 100644 --- a/src/nav/root.yml +++ b/src/nav/root.yml @@ -54,6 +54,8 @@ pages: path: nrql - title: Service Level Management path: service-level-management + - title: Scorecards + path: scorecards - title: Security - title: New Relic IAST path: iast diff --git a/src/nav/scorecards.yml b/src/nav/scorecards.yml new file mode 100644 index 00000000000..1bd65c574ff --- /dev/null +++ b/src/nav/scorecards.yml @@ -0,0 +1,20 @@ +title: Scorecards +path: /docs/scorecards +pages: + - title: Scorecards overview + path: /docs/scorecards/scorecards + - title: Scorecards rules + path: /docs/scorecards/scorecards-rules + - title: Single Scorecard view + path: /docs/scorecards/single-scorecard-view + - title: Rule view + path: /docs/scorecards/rule-view + - title: Teams view + path: /docs/scorecards/teams-view + - title: Scorecards API + path: /docs/scorecards/scorecards-api + - title: Send data to NR to build rules + path: /docs/scorecards/send-data-to-nr-to-build-rules + + + \ No newline at end of file diff --git a/static/images/rules-view.webp b/static/images/rules-view.webp new file mode 100644 index 00000000000..f419f53ec84 Binary files /dev/null and b/static/images/rules-view.webp differ diff --git a/static/images/scorecard-overview-page.webp b/static/images/scorecard-overview-page.webp new file mode 100644 index 00000000000..13ac1c3f781 Binary files /dev/null and b/static/images/scorecard-overview-page.webp differ diff --git a/static/images/single-scorecard-view.webp b/static/images/single-scorecard-view.webp new file mode 100644 index 00000000000..982e56a94ac Binary files /dev/null and b/static/images/single-scorecard-view.webp differ diff --git a/static/images/teams-view.webp b/static/images/teams-view.webp new file mode 100644 index 00000000000..13ac1c3f781 Binary files /dev/null and b/static/images/teams-view.webp differ