Added new AWS Attributes for various services #1794

AsakerMohd · 2025-01-23T06:26:02Z

Changes

Added new AWS Attributes for various services including SQS, SNS, Step Functions, Bedrock as well as a new attribute for Lambda.

Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.

Merge requirement checklist

CONTRIBUTING.md guidelines followed.
Change log entry added, according to the guidelines in When to add a changelog entry.
- If your PR does not need a change log, start the PR title with [chore]
schema-next.yaml updated with changes to existing conventions.

model/aws/registry.yaml

lmolkova

Please review existing semantic conventions:

there are several existing attributes that should be used instead of AAWS-specific ones
some attributes should be defined as generic ones

We avoid defining attributes alone without also defining spans, metrics, events, or resources that reference them, so please add corresponding signal definitions

model/aws/registry.yaml

lmolkova · 2025-01-28T01:52:17Z

model/aws/registry.yaml

+        brief: >
+          The unique identifier of the AWS Bedrock Guardrail. A [guardrail](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) helps safeguard and prevent unwanted behavior from model responses or user messages.
+        examples: ["sgi5gkybzqak"]
+      - id: aws.bedrock.knowledge_base.id


I think we'd need a generic attribute for this one, but it's not defined yet. I'd be fine with adding one, but it's likely to be deprecated in the future.

If the generic one is likely to be deprecated in the future, I'm in favor of keeping these as AWS specific ones. Does that work?

lmolkova · 2025-01-28T01:56:24Z

model/aws/registry.yaml

+          The name of the AWS Kinesis [stream](https://docs.aws.amazon.com/streams/latest/dev/introduction.html) the request refers to. Corresponds to the `--stream-name` parameter of the
+          Kinesis [describe-stream](https://docs.aws.amazon.com/cli/latest/reference/kinesis/describe-stream.html) operation.
+        examples: ["some-stream-name"]
+  - id: registry.aws.stepfunctions


cc @open-telemetry/lambda-extension-approvers

lmolkova · 2025-01-28T01:58:40Z

model/aws/registry.yaml

+    brief: >
+        This document defines attributes for AWS SQS.
+    attributes:
+      - id: aws.sqs.queue.url


we record server.address (host), server.port (port) and messaging.destination.name (queue name) in generic messaging semantic conventions, would url be still useful in this case?

I can probably still use messaging.destination.name in this case.

Taking a look at our use case, we can end up having both the queue URL and the queue name. Using the same attribute for both will not work in this case :(

lmolkova · 2025-01-28T02:01:14Z

model/aws/registry.yaml

@@ -481,3 +488,96 @@ groups:
          and [upload-part-copy](https://docs.aws.amazon.com/cli/latest/reference/s3api/upload-part-copy.html) operations.
          The `part_number` attribute corresponds to the `--part-number` parameter of the
          [upload-part operation within the S3 API](https://docs.aws.amazon.com/cli/latest/reference/s3api/upload-part.html).
+  - id: registry.aws.sqs


if we need any aws.sqs attributes on top of generic messaging, please add a span definition for it and reference all applicable attributes, see kafka yaml and md conventions as an example.

Will add those as well.

model/aws/registry.yaml

lmolkova · 2025-01-28T02:33:02Z

model/aws/registry.yaml

+        brief: >
+          The ARN of the AWS Step Functions State Machine.
+        examples: ["arn:aws:states:us-east-1:123456789012:stateMachine:myStateMachine:1"]
+  - id: registry.aws.bedrock


cc @open-telemetry/semconv-genai-approvers and specifically @xrmx who's adding bedrock instrumentation in python

https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-python-contrib+bedrock&type=pullrequests

Thanks for pinging me, I think I won't need any of the proposed attributes in my work atm (and so have not much clue about them) but I'll keep an eye on this PR.

lmolkova · 2025-01-28T02:33:32Z

model/aws/registry.yaml

@@ -481,3 +488,96 @@ groups:
          and [upload-part-copy](https://docs.aws.amazon.com/cli/latest/reference/s3api/upload-part-copy.html) operations.
          The `part_number` attribute corresponds to the `--part-number` parameter of the
          [upload-part operation within the S3 API](https://docs.aws.amazon.com/cli/latest/reference/s3api/upload-part.html).
+  - id: registry.aws.sqs
+    type: attribute_group
+    display_name: Amazon SQS Attributes


/cc @open-telemetry/semconv-messaging-approvers

github-actions · 2025-02-13T03:20:54Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

github-actions · 2025-03-02T03:22:13Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

lmolkova · 2025-03-05T01:41:07Z

model/gen-ai/registry.yaml

+        brief: >
+          The unique identifier of the AWS Bedrock Guardrail. A [guardrail](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) helps safeguard and prevent unwanted behavior from model responses or user messages.
+        examples: ["sgi5gkybzqak"]
+      - id: gen_ai.aws.bedrock.knowledge_base.id


we're slowly updating this repo to consistently namespace provider-specific attributes following #1708.

The suggestion here is to remove gen_ai prefix from gen_ai.aws.* attributes - this would keep it consistent with the upcoming formal guidance and also with existing aws attributes that don't have any prefix.

lmolkova · 2025-03-05T02:00:12Z

model/gen-ai/registry.yaml

@@ -218,6 +218,11 @@ groups:
            Client-side operations are actions taken on the user's end or within the client application.
          Datastore: A tool used by the agent to access and query structured or unstructured external data for retrieval-augmented tasks or knowledge updates.
        examples: ['function', 'extension', 'datastore']
+      - id: gen_ai.data_source.id


Documenting quick research on the terminology used by others:

Google calls them data stores

OpenAI uses vector stores,

Azure OpenAI uses data sources

Llamaindex calls them data sources

So gen_ai.data_source.id sounds good to me.

lmolkova · 2025-03-05T02:10:27Z

model/gen-ai/registry.yaml

+      - id: gen_ai.data_source.id
+        stability: development
+        type: string
+        brief: The data source identifier.


could we explain it a bit more? E.g. something along these lines:

Suggested change

brief: The data source identifier.

brief: The data source identifier.

note: Data sources are used by AI agents and RAG applications to store grounding data. A data source may be an external database, object store, document collection, website, or any other storage system used by the GenAI agent or application. The `gen_ai.data_source.id` SHOULD match the identifier used by the GenAI system rather than a name specific to the external storage, such as a database or object store. Semantic conventions referencing `gen_ai.data_source.id` MAY also leverage additional attributes, such as `db.*`, to further identify and describe the data source.

lmolkova · 2025-03-05T02:13:28Z

model/gen-ai/registry.yaml

+        brief: >
+          The unique identifier of the AWS Bedrock Guardrail. A [guardrail](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) helps safeguard and prevent unwanted behavior from model responses or user messages.
+        examples: ["sgi5gkybzqak"]
+      - id: gen_ai.aws.bedrock.knowledge_base.id


is there a difference between knowledge_base and data_source? do we need both ids?

From AWS doc "A data source contains files or content with information that can be retrieved when your knowledge base is queried.", so two different ids sounds legit to track the source of the data when you have more than one data source in your knowledge base (up to 5 from a 2024 post).

lmolkova · 2025-03-05T02:18:45Z

model/aws/sdk-spans.yml

+        requirement_level: recommended
+
+  - id: span.aws.sns.client
+    extends: span.aws.client


I don't think the result looks right in the markdown - the span does not follow messaging semantic conventions. Take a look at other messaging spans - https://github.com/open-telemetry/semantic-conventions/tree/main/docs/messaging they inherit from one of the messaging groups and include typical messaging attributes. Here's e.g. kafka yaml

lmolkova · 2025-03-05T02:26:54Z

model/aws/sdk-spans.yml

+
+  - id: span.aws.stepfunctions.client
+    extends: span.aws.client
+    type: span


do you need those defined as spans? I don't think any of them is referenced in markdown and given "Attributes that exist for Step Functions request types. explanation, it sounds like this group does not describe any span.

If you want to document them as spans, could you please document them in markdown including span name format and link from aws-sdk page?

lmolkova · 2025-03-05T02:30:20Z

model/gen-ai/spans.yaml

+        requirement_level: required
+      - ref: gen_ai.aws.bedrock.knowledge_base.id
+        requirement_level: recommended
+      - ref: gen_ai.data_source.id


I think it should be referenced in span.gen_ai.client.create_agent since it's applicable to other agents, not just bedrock

lmolkova · 2025-03-05T02:30:59Z

docs/faas/aws-lambda.md

@@ -52,6 +52,7 @@ and the [cloud resource conventions][cloud]. The following AWS Lambda-specific a
 | Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
 |---|---|---|---|---|---|
 | [`aws.lambda.invoked_arn`](/docs/attributes-registry/aws.md) | string | The full invoked ARN as provided on the `Context` passed to the function (`Lambda-Runtime-Invoked-Function-Arn` header on the `/runtime/invocation/next` applicable). [1] | `arn:aws:lambda:us-east-1:123456:function:myfunction:myalias` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`aws.lambda.resource_mapping.id`](/docs/attributes-registry/aws.md) | string | The UUID of the [AWS Lambda EvenSource Mapping](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html). An event source is mapped to a lambda function. It's contents are read by Lambda and used to trigger a function. | `587ad24b-03b9-4413-8202-bbd56b36e5b7` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |


cc @open-telemetry/lambda-extension-approvers could you please take a look?

lmolkova · 2025-03-05T02:31:56Z

docs/gen-ai/aws-bedrock.md

+linkTitle: AWS Bedrock
+--->
+
+# Semantic conventions for AWS Bedrock operations


please add a link to this (and other new AWS docs) from https://github.com/open-telemetry/semantic-conventions/blob/78c42c576a25743902192466cf7ff81889bf3630/docs/cloud-providers/aws-sdk.md#aws-service-specific-attributes

lmolkova · 2025-03-05T02:33:49Z

model/aws/sdk-spans.yml

@@ -297,3 +297,57 @@ groups:
        requirement_level: recommended
      - ref: aws.s3.part_number
        requirement_level: recommended
+
+  - id: span.aws.sqs.client
+    extends: span.aws.client


JFYI: there is an active issue against this span definition - #1545, from semconv perspective we'd like to fix it at some point which may cause breaking changes if you use rpc.* attributes in AWS SDK instrumentations.

would appreciate your input on this.

Added new AWS Attributes for various services

7244b19

AsakerMohd requested review from a team as code owners January 23, 2025 06:26

Added changelog file

86047e3

github-actions bot added the enhancement New feature or request label Jan 23, 2025

ppittle suggested changes Jan 24, 2025

View reviewed changes

model/aws/registry.yaml Show resolved Hide resolved

ppittle mentioned this pull request Jan 24, 2025

Added AWS SDK Changes to support Bedrock spans and Application Signals open-telemetry/opentelemetry-dotnet-contrib#2458

Closed

4 tasks

lmolkova requested changes Jan 28, 2025

View reviewed changes

lmolkova reviewed Jan 28, 2025

View reviewed changes

github-actions bot added Stale and removed Stale labels Feb 13, 2025

AsakerMohd added 2 commits February 13, 2025 23:07

Merge branch 'main' into add-new-aws-attributes

2d95e15

Added new span attributes and applied comments

ab863ca

AsakerMohd requested review from a team as code owners February 14, 2025 08:31

Merge branch 'main' into add-new-aws-attributes

35910b7

github-actions bot added the Stale label Mar 2, 2025

Merge branch 'main' into add-new-aws-attributes

e23bb5b

lmolkova reviewed Mar 5, 2025

View reviewed changes

github-actions bot removed the Stale label Mar 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added new AWS Attributes for various services #1794

Added new AWS Attributes for various services #1794

AsakerMohd commented Jan 23, 2025 •

edited

Loading

lmolkova left a comment

lmolkova Jan 28, 2025

AsakerMohd Feb 14, 2025

lmolkova Jan 28, 2025

lmolkova Jan 28, 2025

AsakerMohd Feb 14, 2025

AsakerMohd Feb 14, 2025

lmolkova Jan 28, 2025

AsakerMohd Feb 14, 2025 •

edited

Loading

lmolkova Jan 28, 2025

xrmx Jan 28, 2025

lmolkova Jan 28, 2025

github-actions bot commented Feb 13, 2025

github-actions bot commented Mar 2, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

xrmx Mar 5, 2025 •

edited

Loading

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

lmolkova Mar 5, 2025

	brief: The data source identifier.
	brief: The data source identifier.
	note: Data sources are used by AI agents and RAG applications to store grounding data. A data source may be an external database, object store, document collection, website, or any other storage system used by the GenAI agent or application. The `gen_ai.data_source.id` SHOULD match the identifier used by the GenAI system rather than a name specific to the external storage, such as a database or object store. Semantic conventions referencing `gen_ai.data_source.id` MAY also leverage additional attributes, such as `db.*`, to further identify and describe the data source.

Added new AWS Attributes for various services #1794

Are you sure you want to change the base?

Added new AWS Attributes for various services #1794

Conversation

AsakerMohd commented Jan 23, 2025 • edited Loading

Changes

Merge requirement checklist

lmolkova left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AsakerMohd Feb 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 13, 2025

github-actions bot commented Mar 2, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xrmx Mar 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AsakerMohd commented Jan 23, 2025 •

edited

Loading

AsakerMohd Feb 14, 2025 •

edited

Loading

xrmx Mar 5, 2025 •

edited

Loading