Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable prompt extensions in SWE-bench #6391

Merged
merged 2 commits into from
Jan 21, 2025
Merged

Disable prompt extensions in SWE-bench #6391

merged 2 commits into from
Jan 21, 2025

Conversation

enyst
Copy link
Collaborator

@enyst enyst commented Jan 21, 2025

End-user friendly description of the problem this fixes or functionality that this introduces

  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

Disable prompt extension in SWE-bench. (not sure since when it was enabled, 5 days ago it was enabled)

Otherwise, the prompt can currently include (for astropy-13033):

"<extra_info>\nThe following information has been included based on a keyword match for \"github\". It may or may not be relevant to the user's request.\n\nYou have access to an environment variable, `GITHUB_TOKEN`, which allows you to interact with\nthe GitHub API.\n\nYou can use `curl` with the `GITHUB_TOKEN` to interact with GitHub's API.\nALWAYS use the GitHub API for operations instead of a web browser.\n\nHere are some instructions for pushing, but ONLY do this if the user asks you to:\n* NEVER push directly to the `main` or `master` branch\n* Git config (username and email) is pre-set. Do not modify.\n* You may already be on a branch starting with `openhands-workspace`. Create a new branch with a better name before pushing.\n* Use the GitHub API to create a pull request, if you haven't already\n* Once you've created your own branch or a pull request, continue to update it. Do NOT create a new one unless you are explicitly asked to. Update the PR title and description as necessary, but don't change the branch name.\n* Use the main branch as the base branch, unless the user requests otherwise\n* After opening or updating a pull request, send the user a short message with a link to the pull request.\n* Prefer \"Draft\" pull requests when possible\n* Do all of the above in as few steps as possible. E.g. you could open a PR with one step by running the following bash commands:\n```bash\ngit remote -v && git branch # to find the current org, repo and branch\ngit checkout -b create-widget && git add . && git commit -m \"Create widget\" && git push -u origin create-widget\ncurl -X POST \"https://api.github.com/repos/$ORG_NAME/$REPO_NAME/pulls\" \\\n    -H \"Authorization: Bearer $GITHUB_TOKEN\" \\\n    -d '{\"title\":\"Create widget\",\"head\":\"create-widget\",\"base\":\"openhands-workspace\"}'\n```\n</extra_info>"

Link of any specific issues this addresses


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:b526365-nikolaik   --name openhands-app-b526365   docker.all-hands.dev/all-hands-ai/openhands:b526365

@enyst enyst requested a review from xingyaoww January 21, 2025 16:43
Copy link
Collaborator

@xingyaoww xingyaoww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we disabled it... thanks for catching that

@enyst enyst enabled auto-merge (squash) January 21, 2025 17:11
@enyst enyst merged commit 5b7fcfb into main Jan 21, 2025
16 checks passed
@enyst enyst deleted the enyst/disable-prompt-ext branch January 21, 2025 17:18
zchn pushed a commit to zchn/OpenHands that referenced this pull request Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants