Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: DIA-1402: V1-Submit Prompt auto-refinement job #27

Merged
merged 12 commits into from
Oct 1, 2024

Conversation

matt-bernstein
Copy link
Contributor

Plan for autorefinement:

# this PR - DIA-1402/1407
# makes a single api call to the teacher model and returns the new prompt version immediately
# can extend in future to update the current prompt version using PATCH instead of POST
# other parameters: teacher model provider connection id, teacher model name, project id
/api/prompts/{id}/versions/{version_id}/refine-prompt

# autorefinement using ground truth data
# creates a new "run" object analogous to an InferenceRun, the new or updated prompt version and new inference runs are available when the RefinementRun completes
# same parameters: teacher model provider connection id, teacher model name, project id
# manage state the same way as with InferenceRuns using polling, or try something new like websockets?
/api/prompts/{id}/versions/{version_id}/refinement-runs/{run_id}

# create a prompt version with the "prompt" field already filled in
# extend existing create endpoint
# parameters: optional[teacher model provider connection id, teacher model name], optional[project id]
# if teacher provided, ask it for a good prompt given the skill
# if project id provided, enrich the teacher info with example tasks from the project
# if optional params not provided, "prompt" field can still be null - fallback to a generic string template using skill name and input fields
POST /api/prompts/{id}/versions/

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

1 similar comment
@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

1 similar comment
@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

@matt-bernstein matt-bernstein merged commit 6914716 into master Oct 1, 2024
1 check passed
@robot-ci-heartex
Copy link
Collaborator

Follow Merge downstream workflow has been failed.

Workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants