Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MLOB-2240] fix(openai, langchain, llmobs): OpenAI and LangChain instrumentation is ESM-compatible #5267

Merged
merged 11 commits into from
Feb 20, 2025

Conversation

sabrenner
Copy link
Collaborator

@sabrenner sabrenner commented Feb 13, 2025

What does this PR do?

This PR does a few things:

  1. Wraps openai ESM files, with .mjs file extensions
  2. Adds esmFirst: true for langchain and openai modules so that their internals imports trigger IITM hooks
  3. Adds data exclusions for IITM hooks for langchain and openai sub-packages that have known problematic issues (openai, langchain).

Closes #4277

Motivation

Unblocking ESM support for OpenAI (>=4.1) and LangChain instrumentation. Additionally unblocks for LLM Observability.

Copy link

github-actions bot commented Feb 13, 2025

Overall package size

Self size: 8.77 MB
Deduped: 94.97 MB
No deduping: 95.48 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.5.1 | 9.79 MB | 10.17 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 835.4 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Copy link

codecov bot commented Feb 13, 2025

Codecov Report

Attention: Patch coverage is 94.28571% with 2 lines in your changes missing coverage. Please review.

Project coverage is 80.94%. Comparing base (23abc09) to head (22cec44).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
packages/datadog-instrumentations/src/openai.js 94.28% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5267      +/-   ##
==========================================
+ Coverage   80.59%   80.94%   +0.34%     
==========================================
  Files         477      488      +11     
  Lines       21463    21844     +381     
==========================================
+ Hits        17298    17681     +383     
+ Misses       4165     4163       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sabrenner sabrenner force-pushed the sabrenner/llmobs-integrations-esm branch from fa3b9eb to 2cd68ea Compare February 13, 2025 19:22
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Feb 13, 2025

Datadog Report

Branch report: sabrenner/llmobs-integrations-esm
Commit report: 2e3dc63
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 673 Passed, 0 Skipped, 15m 49.46s Total Time

@pr-commenter
Copy link

pr-commenter bot commented Feb 13, 2025

Benchmarks

Benchmark execution time: 2025-02-20 14:41:47

Comparing candidate commit 22cec44 in PR branch sabrenner/llmobs-integrations-esm with baseline commit 23abc09 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 919 metrics, 14 unstable metrics.

@sabrenner sabrenner closed this Feb 13, 2025
@sabrenner sabrenner reopened this Feb 13, 2025
@sabrenner sabrenner marked this pull request as ready for review February 13, 2025 20:05
@sabrenner sabrenner requested review from a team as code owners February 13, 2025 20:05
@sabrenner sabrenner force-pushed the sabrenner/llmobs-integrations-esm branch from d8a20e2 to 319074c Compare February 13, 2025 21:24
@sabrenner sabrenner requested a review from bengl February 19, 2025 14:38
bengl
bengl previously approved these changes Feb 20, 2025
@sabrenner sabrenner merged commit 96bb84a into master Feb 20, 2025
395 checks passed
@sabrenner sabrenner deleted the sabrenner/llmobs-integrations-esm branch February 20, 2025 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error with ESM support when using the openai package
2 participants