feat: Extend EventStreamRuntime to support attaching to running containers

[qiangli]

End-user friendly description of the problem this fixes or functionality that this introduces

Modify the existing EventStreamRuntime to support attaching to existing running docker containers.
Provide working example files and instructions for building and running custom docker sandbox using the new EventStreamRuntime feature.

• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

---

Give a summary of what the PR does, explaining any non-trivial design decisions

This PR introduces two new optional sandbox configuration attributes with minimal modification to the existing code base for building and running custom sandboxes.

• container_name: The name of the container.
• docker_endpoint: The context endpoint for the sandbox Docker daemon.

These two new attributes are used to attach to the running container and the existing remote_runtime_api_url attribute is used to communicate with the running container.

The new feature could empower developers to build sandbox with any features that Docker supports without constraints and it would better fit developers' daily workflows.

EventStreamRuntime will now support local and remote docker sandbox over socket/tcp (this is an out of the box feature of docker). see https://docs.docker.com/engine/manage-resources/contexts/

---

Link of any specific issues this addresses

[rbren]

@qiangli I'm unclear about how this is different from the EventStreamRuntime, which runs inside of docker, using the same server. Can you explain what the goal is here, and why the current solutions don't meet your needs?

[rbren]

see comment above

[mamoodi]

Going to close this for now since it's been a while since it was opened up and no response to the comment. Please feel free to open up a PR if there is a valid answer to the comment and worth pursuing!

[qiangli]

My apology. I did not see your comments until now.

I have been trying to integrate openhands into my daily workflow as a developer and my requirement is simple: the ability to add volumes, envs, privileged/cap_add as a minimum for the sandbox separate from the main openhands processes/config. Ideally, all the docker options are supported once and for all: https://docs.docker.com/reference/cli/docker/container/create/

My earlier PR #4368 and this one tried to provide a solution.

This PR duplicated the code from EventStreamRuntime but with one big difference - this new docker runtime does not create/start the sandbox container - which is completely left to the developer so the developer has 100% control over the creation/starting/stopping of the sandbox locally on the same host as openhands or different hosts remotely.

In the end, hopefully with this change, I could setup and run openhands and have it point to a instance on my local dev machine or a remote test/stage environment. btw, it is very unlikely I could convince the team to install openhands and docker on the remote machines - not at least on every one of them.

[mamoodi]

Reopening. @rbren please see new comment

[rbren]

Ahh what if we simply add these options to the existing EventStreamRuntime? that will be much easier to maintain

[rbren]

this seems like an unnecessary assumption--isn't it possible someone would want to set the docker endpoint without this behavior?

[qiangli]

Not as is currently implemented. This is a new configuration attribute.

[qiangli]

A good point that docker_endpoint can be used and managed by openhands in future for remote instances.
Instroducing a new config attribute attach_to_existing to make it explicit.

[rbren]

I think the goal here is:

"I want to start a single docker container, and have ALL my openhands sessions use that running container"

This is maybe a little dangerous, and potentially problematic with future work, where we'll allow users to have multiple concurrent OH sessions. In that case, you'd have 10 agents all clobbering the same workspace

Based on your last comment I thought the goal was more to have certain kinds of privileges on the containers OH creates. Could we add that as configuration (e.g. attaching volumes or running as privileged) to the EventStreamRuntime?

[qiangli]

"I want to start a single docker container, and have ALL my openhands sessions use that running container"

It is 1:1. We could make it by session, i.e. each session pointing to different existing running containers. Unless I'm missing something, the current implementation of workspace/mount path in the config.toml is also assuming 1:1?

[qiangli]

will open a separate PR or maybe reinstate #4368