can not initializing the RouteLLM controller

[Walter-Liu-X]

when i initializing the RouteLLM controller with demo, i meet the errors below:

import os
from routellm.controller import Controller

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxx"
os.environ["ANYSCALE_API_KEY"] = "esecret_xxxxx"

client = Controller(
  routers=["mf"],
  strong_model="gpt-4-1106-preview",
  weak_model="anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1",
)

response = client.chat.completions.create(
  model="router-mf-0.11593",
  messages=[
    {"role": "user", "content": "Hello!"}
  ]
)

Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/openai_server.py", line 22, in <module>
    from routellm.controller import Controller, RoutingError
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/controller.py", line 10, in <module>
    from routellm.routers.routers import ROUTER_CLS
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/routers.py", line 17, in <module>
    from routellm.routers.matrix_factorization.model import MODEL_IDS, MFModel
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/matrix_factorization/model.py", line 4, in <module>
    from routellm.routers.similarity_weighted.utils import OPENAI_CLIENT
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/similarity_weighted/utils.py", line 8, in <module>
    from sklearn.linear_model import LogisticRegression
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/__init__.py", line 84, in <module>
    from .base import clone
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/base.py", line 19, in <module>
    from .utils._estimator_html_repr import _HTMLDocumentationLinkMixin, estimator_html_repr
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/__init__.py", line 11, in <module>
    from ._chunking import gen_batches, gen_even_slices
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_chunking.py", line 8, in <module>
    from ._param_validation import Interval, validate_params
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_param_validation.py", line 14, in <module>
    from .validation import _is_arraylike_not_scalar
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/validation.py", line 26, in <module>
    from ..utils._array_api import _asarray_with_order, _is_numpy_namespace, get_namespace
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_array_api.py", line 11, in <module>
    from .fixes import parse_version
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/fixes.py", line 20, in <module>
    import scipy.stats
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/__init__.py", line 606, in <module>
    from ._stats_py import *
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/_stats_py.py", line 49, in <module>
    from . import distributions
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/distributions.py", line 11, in <module>
    from . import _discrete_distns
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/_discrete_distns.py", line 10, in <module>
    from scipy.interpolate import interp1d
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/__init__.py", line 167, in <module>
    from ._interpolate import *
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_interpolate.py", line 14, in <module>
    from . import _fitpack_py
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_fitpack_py.py", line 8, in <module>
    from ._fitpack_impl import bisplrep, bisplev, dblint  # noqa: F401
  File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_fitpack_impl.py", line 103, in <module>
    'iwrk': array([], dfitpack_int), 'u': array([], float),
TypeError

i use python 3.9.5 and install routellm with

python3 -m venv .routellm
source .routellm/bin/activate

pip install "routellm[serve,eval]"

pls help to find the problem, ths!

[iojw]

Looks like a dependency issue. Could you share the output of pip list?

[Walter-Liu-X]

python3.9 environment pip freeze result:

aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.7.4
charset-normalizer==3.3.2
click==8.1.7
contourpy==1.2.1
cycler==0.12.1
datasets==2.20.0
dill==0.3.8
distro==1.9.0
dnspython==2.6.1
email-validator==2.2.0
exceptiongroup==1.2.1
fastapi==0.111.0
fastapi-cli==0.0.4
filelock==3.15.4
fonttools==4.53.1
frozenlist==1.4.1
fsspec==2024.5.0
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
huggingface-hub==0.23.4
idna==3.7
importlib-metadata==8.0.0
importlib-resources==6.4.0
jinja2==3.1.4
joblib==1.4.2
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
kiwisolver==1.4.5
litellm==1.41.14
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.9.1
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.2.1
numpy==2.0.0
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.5.82
nvidia-nvtx-cu12==12.1.105
openai==1.35.12
orjson==3.10.6
packaging==24.1
pandarallel==1.6.5
pandas==2.2.2
pillow==10.4.0
psutil==6.0.0
pyarrow==16.1.0
pyarrow-hotfix==0.6
pydantic==2.8.2
pydantic-core==2.20.1
pygments==2.18.0
pyparsing==3.1.2
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
referencing==0.35.1
regex==2024.5.15
requests==2.32.3
rich==13.7.1
routellm==0.2.0
rpds-py==0.19.0
safetensors==0.4.3
scikit-learn==1.5.1
scipy==1.13.1
sglang==0.1.19
shellingham==1.5.4
shortuuid==1.0.13
six==1.16.0
sniffio==1.3.1
starlette==0.37.2
sympy==1.13.0
threadpoolctl==3.5.0
tiktoken==0.7.0
tokenizers==0.19.1
torch==2.3.1
tqdm==4.66.4
transformers==4.42.3
triton==2.3.1
typer==0.12.3
typing-extensions==4.12.2
tzdata==2024.1
ujson==5.10.0
urllib3==2.2.2
uvicorn==0.30.1
xxhash==3.4.1
yarl==1.9.4
zipp==3.19.2

when i use conda create python 3.11 environment and install dependency, it runs normally!

my os is: ubuntu 20.04

[iojw]

I see, thank you for checking!

I suspect this is probably to do some dependency conflict with Python 3.9 then, so I would suggest to use Python >=3.10 for the time being!

[collindutter]

@Walter-Liu-X This is an issue with the recently released numpy==2.0.0. Installing numpy==1.26.4 should resolve the issue.