Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: ollama not wroking #3960

Closed
2 tasks done
kittenvr opened this issue Sep 19, 2024 · 15 comments
Closed
2 tasks done

[Bug]: ollama not wroking #3960

kittenvr opened this issue Sep 19, 2024 · 15 comments
Labels
bug Something isn't working severity:low Minor issues or affecting single user Stale Inactive for 30 days

Comments

@kittenvr
Copy link

Is there an existing issue for the same bug?

Describe the bug

when i start openhands with ollama it just doesnt work

Current OpenHands version

ghcr.io/all-hands-ai/openhands

Installation and Configuration

ollama serve

# The directory you want OpenHands to modify. MUST be an absolute path!
export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://host.docker.internal:11434" \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:main

Model and Agent

model: deepseek-coder-v2:latest

Operating System

linux chachyos

Reproduction Steps

i dont know how to reproduce on other installs

Logs, Errors, Screenshots, and Additional Context

image
https://discord.com/channels/1222935860639563850/1281641914210455643

@kittenvr kittenvr added the bug Something isn't working label Sep 19, 2024
@enyst
Copy link
Collaborator

enyst commented Sep 19, 2024

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

@kittenvr
Copy link
Author

Can please you try this: open the app on localhost:3000, and enter in Settings the model, API key, and base URL? Does it work to see the openhands interface and enter these settings?

https://www.logpasta.com/paste/99927887-c724-425d-9ae5-07580bc64e13

@enyst
Copy link
Collaborator

enyst commented Sep 20, 2024

I think the docker problem is the main issue there, but first, please also try to simplify the command a bit, and pull the 0.9 tag instead of main.

export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -it \
    --pull=always \
    --add-host host.docker.internal:host-gateway \
    -e SANDBOX_USER_ID=$(id -u) \
    -e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    ghcr.io/all-hands-ai/openhands:0.9

About the docker issue: can you try to check if docker networking is working? Could start by restarting Docker and checking if it's up to date. That error doesn't seem related to openhands, but docker installation or configuration, I would say.

@mamoodi mamoodi added the severity:low Minor issues or affecting single user label Oct 2, 2024
@ParkerrDev
Copy link

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

@kittenvr
Copy link
Author

kittenvr commented Nov 1, 2024

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

thanks but im not using ollama anymore, could you check this one out tho #4586 (comment)

@jimmyzhao-57blocks
Copy link

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

when I add --network host, I can't open the localhost:3000 in the browser.

@BridgerB
Copy link

BridgerB commented Nov 9, 2024

It works for me when I add --network host \ on 0.12

Commands I used to get it working:

ollama serve

docker pull docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik

sudo docker run -it --privileged --rm --pull=always \
    --network host \
    -e LLM_API_KEY="ollama" \
    -e LLM_BASE_URL="http://127.0.0.1:11434" \
    -e LLM_OLLAMA_BASE_URL="http://127.0.0.1:11434" \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.12

This is not working for me.
I am able to use

ollama run llama3.1

or

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "2+2?",             
  "stream": false
}'

But when I try your command I get this:

INFO:     Started server process [10]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO:     127.0.0.1:55928 - "GET /api/options/models HTTP/1.1" 200 OK
03:41:36 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55932) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed
INFO:     127.0.0.1:55928 - "GET /api/options/agents HTTP/1.1" 200 OK
INFO:     127.0.0.1:55928 - "GET /api/options/security-analyzers HTTP/1.1" 200 OK
03:41:39 - openhands:ERROR: listen.py:311 - Expected 3 websocket protocols, got 1
INFO:     ('127.0.0.1', 55940) - "WebSocket /ws" 403
INFO:     connection rejected (403 Forbidden)
INFO:     connection closed

@skylarmb
Copy link

skylarmb commented Nov 13, 2024

This is basic docker networking configuration. localhost and 127.0.0.1 refer to addresses within the docker network, not addresses on the host machine. This works fine with the docker command in the docs as long as you correctly specify the docker host as the location of your ollama server.

ollama serve
# from the docs, no modifications required
docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.13-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    -e LOG_ALL_EVENTS=true \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.13

Now open the UI and configure your provider using host.docker.internal to reference the docker host machine network. You don't need to specify an API key.

image

Works fine for me!

@BioCruz-sudo
Copy link

This is basic docker networking configuration. localhost and 127.0.0.1 refer to addresses within the docker network, not addresses on the host machine. This works fine with the docker command in the docs as long as you correctly specify the docker host as the location of your ollama server.

ollama serve
# from the docs, no modifications required
docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.13-nikolaik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    -e LOG_ALL_EVENTS=true \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.13

Now open the UI and configure your provider using host.docker.internal to reference the docker host machine network. You don't need to specify an API key.

image

Works fine for me!

You are AMAZING! Thanks for helping me get this figured out, I got it working on my PC. A bit slow but hey at least it's free! Thanks a bunch :)

@enyst
Copy link
Collaborator

enyst commented Nov 14, 2024

@mamoodi what do you think, could we add @skylarmb 's screenshot to our docs? You know that saying, an image is worth a thousand words!

@mamoodi
Copy link
Collaborator

mamoodi commented Nov 14, 2024

That's great. I've queued this in my todo list and see in incorporating it. Thank you @skylarmb!!!

@TomLucidor
Copy link

TomLucidor commented Nov 20, 2024

@skylarmb should this type of Advanced usage be documented then, since the network configs are mostly trivial? (and yes, that includes adding custom Ollama models like Qwen and Phi that are usually not in the default selector)

@enyst
Copy link
Collaborator

enyst commented Nov 21, 2024

@skylarmb should this type of Advanced usage be documented then, since the network configs are mostly trivial? (and yes, that includes adding custom Ollama models like Qwen and Phi that are usually not in the default selector)

You may want to check this out: https://docs.all-hands.dev/modules/usage/llms/local-llms

Copy link
Contributor

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale Inactive for 30 days label Dec 30, 2024
Copy link
Contributor

github-actions bot commented Jan 6, 2025

This issue was closed because it has been stalled for over 30 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working severity:low Minor issues or affecting single user Stale Inactive for 30 days
Projects
None yet
Development

No branches or pull requests

9 participants