Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(feat): Prompt engineering to remind o1 to generate a patch #4807

Merged
merged 36 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
76cdcd1
Updated tests
AlexCuadron Oct 15, 2024
3beaf5c
chore(deps): bump litellm from 1.49.3 to 1.49.4 (#4406)
dependabot[bot] Oct 15, 2024
c8db8aa
chore(deps-dev): bump llama-index from 0.11.17 to 0.11.18 (#4408)
dependabot[bot] Oct 15, 2024
308dc62
chore(deps): bump modal from 0.64.181 to 0.64.182 (#4407)
dependabot[bot] Oct 15, 2024
158a923
refactor: move get_pairs from memory to shared utils (#4411)
xingyaoww Oct 15, 2024
b6a9163
Fix eval output path in case of @ char (#4416)
mamoodi Oct 15, 2024
8ba531a
Fix for lockup - create the runtime in a background thread (#4412)
tofarr Oct 15, 2024
87f6870
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 16, 2024
6037e20
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 16, 2024
0c5de4c
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 17, 2024
12798fd
Added support to specify the platform on which the image should be bu…
AlexCuadron Oct 19, 2024
ef3646f
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 20, 2024
18bdb56
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 22, 2024
7ca0de6
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 23, 2024
5a76cc8
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 24, 2024
4a7ef31
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 25, 2024
32c69af
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 28, 2024
bf8b4c0
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 28, 2024
e284c95
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 28, 2024
619bbf1
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 30, 2024
65ec945
Merge remote-tracking branch 'upstream/main'
AlexCuadron Oct 31, 2024
d644f45
Merge remote-tracking branch 'upstream/main'
AlexCuadron Nov 5, 2024
ec94128
Merge remote-tracking branch 'upstream/main'
AlexCuadron Nov 5, 2024
064e4ad
prompt engineering to remind o1 to generate a patch
AlexCuadron Nov 6, 2024
b05b47f
Merge branch 'main' into o1
AlexCuadron Nov 6, 2024
aaba2d4
Make SWE Bench specific instructions conditional based on environment…
AlexCuadron Nov 7, 2024
7773fc6
Merge branch 'main' into o1
AlexCuadron Nov 7, 2024
b19fdf1
fix
AlexCuadron Nov 7, 2024
f4b7066
fix
AlexCuadron Nov 7, 2024
af947b0
fix
AlexCuadron Nov 7, 2024
9cf4927
fixed according to comments
AlexCuadron Nov 8, 2024
f76909e
Merge branch 'main' into o1
AlexCuadron Nov 8, 2024
bc9f635
Update run_infer.py
AlexCuadron Nov 8, 2024
deb54f7
Update run_infer.py
AlexCuadron Nov 8, 2024
e80d170
Merge branch 'main' into o1
AlexCuadron Nov 8, 2024
1033d61
fixed based on comments
AlexCuadron Nov 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions evaluation/swe_bench/run_infer.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,14 @@ def get_instruction(instance: pd.Series, metadata: EvalMetadata):
"Your thinking should be thorough and so it's fine if it's very long.\n"
)

instruction += """
<IMPORTANT>
- You MUST generate only one action per turn!
- A patch is a set of changes to the source code of the codebase that you are given
- You MUST generate a patch that attempts to fix the issue described in the <pr_description>
</IMPORTANT>
"""

if RUN_WITH_BROWSING:
instruction += (
'<IMPORTANT!>\n'
Expand Down
10 changes: 7 additions & 3 deletions openhands/agenthub/codeact_agent/codeact_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -508,8 +508,12 @@ def _get_messages(self, state: State) -> list[Message]:
None,
)
# do not add this for function calling
if latest_user_message:
reminder_text = f'\n\nENVIRONMENT REMINDER: You have {state.max_iterations - state.iteration} turns left to complete the task. When finished reply with <finish></finish>.'
latest_user_message.content.append(TextContent(text=reminder_text))
if not latest_user_message:
return messages

# Build environment reminder text
reminder_text = f'\n\nENVIRONMENT REMINDER: You have {state.max_iterations - state.iteration} turns left to complete the task. When finished reply with <finish></finish>.'

latest_user_message.content.append(TextContent(text=reminder_text))

return messages