Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Models registered via web ui --register file.json don't appear as available on openai server list #3537

Open
surak opened this issue Sep 26, 2024 · 1 comment

Comments

@surak
Copy link
Collaborator

surak commented Sep 26, 2024

If you have a model loaded from a json file, it will appear on the list of the available models on the web UI, but it won't appear on the openai_api_server, because it's not added to the controller.

@surak
Copy link
Collaborator Author

surak commented Sep 26, 2024

Adding the model with the register_worker.py will fail with a 404.

For example, an ollama server is running at its default port 11434. If I add it via the json file, it opens fine on the web ui.
If I try to open it with the python3 fastchat/serve/register_worker.py --controller http://localhost:21001 --worker-name http://localhost:11434, it will fail.

Should we add support for the json file on the register_worker.py or make the web interface add the worker to the controller?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant