Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify draft llm #6281

Merged
merged 14 commits into from
Jan 17, 2025
Merged

Simplify draft llm #6281

merged 14 commits into from
Jan 17, 2025

Conversation

enyst
Copy link
Collaborator

@enyst enyst commented Jan 15, 2025

End-user friendly description of the problem this fixes or functionality that this introduces

  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below
    Users can now define a different LLM as a custom LLM config named with a predefined name: draft_editor. To use it for edit drafts, you can, as before, set the agent option to use llm editor (codeact_enable_llm_editor).

Give a summary of what the PR does, explaining any non-trivial design decisions

This PR proposes to simplify the code handling the draft editor LLM:

  • make it a custom configuration with a specific name: [llm.draft_editor]
    • the user can fill it in with a different model than the main LLM
    • fallback to the main LLM
  • clean up the special handling code, it's no longer special
  • the actual pre-condition remains the same: the LLMBasedFileEditTool is only enabled if it's set in the agent, codeact_enable_llm_editor.

Also bug fix:

  • don't load the draft editor when disabled in config

Note: this was based on top of #4415, the diff will be very simple once that is merged.


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:6a7a772-nikolaik   --name openhands-app-6a7a772   docker.all-hands.dev/all-hands-ai/openhands:6a7a772

@enyst enyst force-pushed the enyst/draft-llm-refactoring branch from de65018 to 8d677c7 Compare January 15, 2025 01:39
@enyst
Copy link
Collaborator Author

enyst commented Jan 15, 2025

@openhands-agent-exp Read the diff of this PR carefully. We need to document the draft_editor like a predefined custom LLM configuration. Check where the documentation for custom configs is, and add it there in an appropriate way. It's the first predefined custom config, but we will have more!

@openhands-agent
Copy link
Contributor

OpenHands started fixing the pr! You can monitor the progress here.

@openhands-agent
Copy link
Contributor

Overview of Changes:

✅ Successfully Resolved:

  • Added draft_editor documentation in appropriate location
  • Created structured "Predefined Custom Configurations" section
  • Provided comprehensive config details including TOML, parameters, and use cases
  • Maintained documentation consistency
  • Established framework for future predefined configs

🔍 Current Status:
All identified issues appear to have been successfully resolved. The changes are thorough and well-structured, addressing both immediate documentation needs and future scalability.

No remaining issues are apparent from the provided information.

@enyst enyst requested a review from xingyaoww January 15, 2025 02:11
The `draft_editor` configuration is optimized for tasks that involve editing and refining code or text. It uses parameters that balance between creativity and precision, making it ideal for tasks like code review suggestions, documentation improvements, and text refinement.

```toml
[llm.draft_editor]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this "predefined" in the sense that the user need not specify anything to use this config? I read this as just an example draft editor configuration.

Copy link
Collaborator Author

@enyst enyst Jan 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually just "predefined name". 🤔 That is, openhands knows what to do with a configuration with this name, if the user fills it in.
Maybe it's not the best word.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Might help to clarify that the given configuration is an example that would automatically be used as the draft editor LLM if provided.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just like the user can define any other custom configuration like [llm.eval-gpt4o] etc, and they can use them or not, use them in an agent, or in general, there are some ... maybe we should call them reserved names? [llm.draft_editor] will be used for the draft llm editor feature.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Reserved names" makes sense, I think that fits the concept. Implies there's some hard-coded logic around the name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copy link
Collaborator

@csmith49 csmith49 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much simpler, I like this approach quite a bit.

@enyst enyst enabled auto-merge (squash) January 17, 2025 17:11
@enyst enyst merged commit 85a760e into main Jan 17, 2025
17 checks passed
@enyst enyst deleted the enyst/draft-llm-refactoring branch January 17, 2025 17:38
csmith49 pushed a commit to csmith49/OpenHands that referenced this pull request Jan 19, 2025
@enyst enyst removed the request for review from xingyaoww January 21, 2025 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants