Skip to content

[Feature Request] Support for llm-emit-token-metric in AIProjectClient ChatCompletionsClient for Azure OpenAI Token Tracking #39385

Closed
@o1100

Description

@o1100

Feature Request: Enable llm-emit-token-metric or azure-openai-emit-token-metric in AIProjectClient's ChatCompletionsClient


Is your feature request related to a problem? Please describe.

I am looking to use the llm-emit-token-metric or azure-openai-emit-token-metric headers within Azure API Management (APIM) to capture token usage. My setup uses Azure Foundry alongside the new azure.ai.projects AIProjectClient and its corresponding ChatCompletionsClient.

While I understand how to enable token metric headers when performing regular POST requests to Azure OpenAI endpoints, I have been unable to find functionality to do this within the ChatCompletionsClient.

The goal is to configure an APIM instance for the endpoint https://<deployment>.openai.azure.com/ and log token metrics to Azure Log Analytics.


Describe the solution you'd like

I would like the ChatCompletionsClient (from azure.ai.projects) to include functionality for enabling and utilizing the llm-emit-token-metric or azure-openai-emit-token-metric headers when interacting with Azure OpenAI services.

This would allow token usage tracking to seamlessly integrate with APIM and Log Analytics while leveraging the new client library.


Describe alternatives you've considered

  • I have been able to use regular POST requests to Azure OpenAI endpoints and successfully capture token metrics with the llm-emit-token-metric or azure-openai-emit-token-metric headers.
  • However, I cannot find similar functionality within the ChatCompletionsClient of azure.ai.projects.

Additional context

  • I have reviewed the Journey of the Geek blog post for guidance on this matter. While their solution effectively captures token metrics, it does not utilize the new AIProjectClient or ChatCompletionsClient from the azure.ai.projects library.

Metadata

Metadata

Assignees

Labels

AI ProjectsClientThis issue points to a problem in the data-plane of the library.Service AttentionWorkflow: This issue is responsible by Azure service team.customer-reportedIssues that are reported by GitHub users external to the Azure organization.feature-requestThis issue requires a new behavior in the product in order be resolved.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions