The checkpoint you are trying to load has model type mllama but Transformers does not recognize this architecture.

[kekslop]

fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | Traceback (most recent call last):
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 989, in from_pretrained
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     config_class = CONFIG_MAPPING[config_dict["model_type"]]
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 691, in __getitem__
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     raise KeyError(key)
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | KeyError: 'mllama'
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | 
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | During handling of the above exception, another exception occurred:
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | 
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | Traceback (most recent call last):
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     return _run_code(code, main_globals, None,
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     exec(code, run_globals)
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/home/FastChat/fastchat/serve/vllm_worker.py", line 290, in <module>
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     engine = AsyncLLMEngine.from_engine_args(engine_args)
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 457, in from_engine_args
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     engine_config = engine_args.create_engine_config()
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 699, in create_engine_config
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     model_config = ModelConfig(
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 153, in __init__
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     self.hf_config = get_config(self.model, trust_remote_code, revision,
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/config.py", line 58, in get_config
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     raise e
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/config.py", line 43, in get_config
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     config = AutoConfig.from_pretrained(
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |   File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 991, in from_pretrained
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr |     raise ValueError(
fastchat-model-worker-1  | 2024-09-26 06:12:44 | ERROR | stderr | ValueError: The checkpoint you are trying to load has model type `mllama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

I am trying to run meta-llama/Llama-3.2-11B-Vision-Instruct, but I understand that support for mllama models is not yet available?

[surak]

The last line gives the hint that maybe the transformers python module is out of date - have you tried updating it?