Skip to content

Commit

Permalink
Update grading functions:
Browse files Browse the repository at this point in the history
1. Fix the gpt-4o grader API version to avoid using the recently released one.
2. Fix grading API calling logics for reasoning Qs to increase robustness.
  • Loading branch information
Colin Wang committed Aug 18, 2024
1 parent 6d225db commit 5e10248
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
2 changes: 1 addition & 1 deletion src/descriptive_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def get_descriptive_result_gpt(client, prompt, length, max_retries=10):
"content": prompt,
}
],
model="gpt-4o",
model="gpt-4o-2024-05-13",
response_format={"type": "json_object"},
n=1,
max_tokens=max_tokens,
Expand Down
3 changes: 2 additions & 1 deletion src/reasoning_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def get_reasoning_result_gpt(client, prompt, max_retries=10):
"content": prompt,
}
],
model="gpt-4o",
model="gpt-4o-2024-05-13",
response_format={"type": "json_object"},
n=1,
max_tokens=max_tokens,
Expand All @@ -33,6 +33,7 @@ def get_reasoning_result_gpt(client, prompt, max_retries=10):
if 'Unterminated string starting at' in str(e):
if max_tokens >= 1024:
print(f"Failed to get response for prompt: {prompt}")
ext, scr = 'Failed to parse response', -1
break
else:
max_tokens = min(1024, max_tokens * 2) # double the max_tokens
Expand Down

0 comments on commit 5e10248

Please sign in to comment.