Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLM Custom Attributes Context Manager API #1214

Merged
merged 32 commits into from
Oct 10, 2024
Merged

Conversation

umaannamalai
Copy link
Contributor

Add new WithLlmCustomAttributes context manager API

Copy link

github-actions bot commented Sep 20, 2024

🦙 MegaLinter status: ✅ SUCCESS

Descriptor Linter Files Fixed Errors Elapsed time
✅ PYTHON bandit 5 0 6.63s
✅ PYTHON black 20 0 0 3.11s
✅ PYTHON flake8 20 0 1.66s
✅ PYTHON isort 20 0 0 0.29s
✅ PYTHON pylint 20 0 11.3s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security

@codecov-commenter
Copy link

codecov-commenter commented Sep 20, 2024

Codecov Report

Attention: Patch coverage is 88.23529% with 4 lines in your changes missing coverage. Please review.

Project coverage is 81.24%. Comparing base (2d17237) to head (182cc71).

Files with missing lines Patch % Lines
newrelic/api/llm_custom_attributes.py 80.00% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1214      +/-   ##
==========================================
+ Coverage   81.20%   81.24%   +0.03%     
==========================================
  Files         197      198       +1     
  Lines       21644    21675      +31     
  Branches     3428     3434       +6     
==========================================
+ Hits        17576    17609      +33     
+ Misses       2937     2935       -2     
  Partials     1131     1131              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hmstepanek hmstepanek self-requested a review September 20, 2024 23:11
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to let you fix the implementation issues first and then I'll take a look at the tests. It looks like there's just some small but large impact issues with the implementation and I think once we fix those the tests will probably be in better shape too.

newrelic/api/llm_custom_attributes.py Outdated Show resolved Hide resolved
newrelic/api/llm_custom_attributes.py Outdated Show resolved Hide resolved
newrelic/api/llm_custom_attributes.py Outdated Show resolved Hide resolved
newrelic/api/llm_custom_attributes.py Outdated Show resolved Hide resolved
newrelic/api/llm_custom_attributes.py Outdated Show resolved Hide resolved
newrelic/hooks/mlmodel_langchain.py Show resolved Hide resolved
newrelic/hooks/mlmodel_openai.py Outdated Show resolved Hide resolved
newrelic/hooks/mlmodel_openai.py Outdated Show resolved Hide resolved
@hmstepanek hmstepanek marked this pull request as ready for review September 25, 2024 21:36
@hmstepanek hmstepanek requested a review from a team as a code owner September 25, 2024 21:36
@mergify mergify bot removed the tests-failing label Sep 25, 2024
@mergify mergify bot removed the tests-failing label Oct 1, 2024
@mergify mergify bot added the tests-failing label Oct 7, 2024
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just use our regular approach of attaching the attrs to the transaction without using contextvars. I don't see that it's needed in this case and I think simpler is better here. Other than that, I just had one minor suggestion for one of the tests. I also committed a small change to undo some unnecessary formatting changes.

tests/agent_features/test_llm_custom_attributes.py Outdated Show resolved Hide resolved
hmstepanek
hmstepanek previously approved these changes Oct 8, 2024
@mergify mergify bot removed the tests-failing label Oct 9, 2024
@mergify mergify bot added the tests-failing label Oct 9, 2024
hmstepanek
hmstepanek previously approved these changes Oct 9, 2024
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple small suggestions about the tests-otherwise looks good!

tests/agent_features/test_llm_custom_attributes.py Outdated Show resolved Hide resolved
@umaannamalai umaannamalai added this to the 10.1.0 milestone Oct 9, 2024
@umaannamalai umaannamalai merged commit 573d5f0 into main Oct 10, 2024
51 of 53 checks passed
@umaannamalai umaannamalai deleted the llm-custom-attrs-api branch October 10, 2024 00:12
@mergify mergify bot removed the tests-failing label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants