-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Chore: clean up LLM (prompt caching, supports fn calling), leftover renames #6095
Conversation
f.write(json.dumps(_d)) | ||
|
||
return resp | ||
except APIError as e: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The diff in llm.py
looks like a lot of change, but it's not: most of it is only indentation, because we are cleaning out this try / except.
@@ -363,7 +363,7 @@ jobs: | |||
image_name=ghcr.io/${{ github.repository_owner }}/runtime:${{ env.RELEVANT_SHA }}-${{ matrix.base_image }} | |||
image_name=$(echo $image_name | tr '[:upper:]' '[:lower:]') | |||
|
|||
TEST_RUNTIME=eventstream \ | |||
TEST_RUNTIME=docker \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is tested in CI and it works just fine, thanks to the compatibility code here it's doing the same thing. This PR just names more things "docker runtime".
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Prompt caching works without the headers now: (Claude Sonnet 3.5)
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks for cleaning this up!
Give a summary of what the PR does, explaining any non-trivial design decisions
This PR does a few clean-ups/leftovers
Also
Link of any specific issues this addresses
Fix #4484
Fix #6085
To run this PR locally, use the following command: