feat(litellm): [MLOB-2326] instrument route request proxy method

[ncybul]

This PR implements APM tracing for LiteLLM's route_request proxy method which is called within both the completion and chat_completion server methods. The host and model name is extracted from the proxy request and attached to the APM span.

I ran the following curl command to give me a trace for calling the chat_completion method.

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful math tutor. Guide the user through the solution step by step."
      },
      {
        "role": "user",
        "content": "how can I solve 8x + 7 = -23"
      }
    ]
}'

I ran the following curl command to give me a trace for calling the completion method (i.e. text completions).

curl -X POST 'http://0.0.0.0:4000/completions' \
-H 'Content-Type: application/json' \
-d '{
    "model": "gpt-3.5-turbo",
    "prompt": "How are you today?"
}'

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

releasenotes/notes/litellm-proxy-tracing-920209d5d51a231b.yaml          @DataDog/apm-python
tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_route_request.json  @DataDog/apm-python
ddtrace/contrib/_litellm.py                                             @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/litellm/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
riotfile.py                                                             @DataDog/apm-python
tests/contrib/litellm/test_litellm.py                                   @DataDog/apm-core-python @DataDog/apm-idm-python
.riot/requirements/143130b.txt                                          @DataDog/apm-python
.riot/requirements/3bb416c.txt                                          @DataDog/apm-python
.riot/requirements/6770e54.txt                                          @DataDog/apm-python
.riot/requirements/739c736.txt                                          @DataDog/apm-python
.riot/requirements/d016adf.txt                                          @DataDog/apm-python

[github-actions]

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 231 ± 2 ms.

The average import time from base is: 233 ± 2 ms.

The import time difference between this PR and base is: -2.25 ± 0.1 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 1.996 ms (0.87%)

ddtrace.bootstrap.sitecustomize 1.322 ms (0.57%)

ddtrace.bootstrap.preload 1.322 ms (0.57%)

ddtrace.internal.remoteconfig.client 0.650 ms (0.28%)

ddtrace 0.674 ms (0.29%)

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-05-13 18:41:08

Comparing candidate commit c1eddeb in PR branch nicole-cybul/instrument-litellm-route-request with baseline commit 690973d in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 522 metrics, 5 unstable metrics.

scenario:iast_aspects-ospathdirname_aspect

• 🟥 execution_time [+818.489ns; +894.899ns] or [+17.399%; +19.023%]

[datadog-datadog-prod-us1]

Datadog Summary

✅ Code Quality    ✅ Code Security    ✅ Dependencies

---

Was this helpful? Give us feedback!