Run GUI and CLI mode at the same time for a single container

[UltraInstinct0x]

What problem or use case are you trying to solve?

I have developed a Discord bot capable of initializing and executing OpenHands Docker containers. My intention is to enable simultaneous execution of graphical user interface (GUI) and command-line interface (CLI) functionalities. This will allow users to interact with the GUI independently, while the bot can simultaneously fulfill tasks through CLI commands, even with voice commands. By incorporating both GUI and CLI interfaces, the functionality of OpenHands can be extended to accommodate these diverse use cases.

Describe the UX of the solution you'd like

Users can utilize speech-to-text or text-to-speech flows to command OpenHands through layers, such as my Discord bot. Subsequently, the layer can execute tasks via CLI, which will impact the GUI itself. This approach enables a genuine collaboration experience.

Do you have thoughts on the technical implementation?

This really shouldn't be a problem with current implementation.

Describe alternatives you've considered

Currently, I am executing two concurrent containers: one for graphical user interface (GUI) and another for command-line interface (CLI). However, these containers lack communication, which may potentially lead to confusion for the end users.

Additional context

[UltraInstinct0x]

I had a look at the codebase and tried to keep a simpler approach by connecting to websocket via a rust exec.
However I see when a repo is selected, i mean nevermind, long story short, websocket connection didn't work as I thought it would cuz there are lots of other processes that needs to run (another container is being run when a repo is selected, i must handle that too with rust workaround.
I'll have another look later on to make this happen with my initial approach.

[UltraInstinct0x]

I’ve made it! I’ll be sharing more details and creating a PR with additional setup instructions on how to run OpenHands like this. Right now, it’s like two agents are controlled from a single one, and even the CLI is running from the first container, so it’s super smooth.

I’ll be working on letting the GUI work with the CLI together and updating the GUI based on the CLI output and states, etc.

[scosenza]

@UltraInstinct0x Any updates or draft PRs you could share? Thanks!

[UltraInstinct0x]

Hey @scosenza, we’re making progress, but I’m not sure I’m comfortable mixing these two from a CLI perspective (which I currently use).

Normally, the GUI would handle runtime containers for you. With my workaround, I define the working directory in advance, along with the LLM and API key. I’m aiming to make the CLI accessible through a GUI container instead. So users can use the GUI as they normally would, but the CLI will also be available.

I’m making some changes and hope they’ll be ready by the end of the week. What do you think?

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue was closed because it has been stalled for over 30 days with no activity.