Open
Description
Hi folk.
I am currently testing on the AWS EC2 G5G series (AWS Graviton2, ARM64).
Here is my error output.
I1122 15:20:46.453111 168 libtorch.cc:2507] TRITONBACKEND_Initialize: pytorch
I1122 15:20:46.453159 168 libtorch.cc:2517] Triton TRITONBACKEND API version: 1.15
I1122 15:20:46.453165 168 libtorch.cc:2523] 'pytorch' TRITONBACKEND API version: 1.15
I1122 15:20:46.453165 167 libtorch.cc:2517] Triton TRITONBACKEND API version: 1.15
I1122 15:20:46.453170 167 libtorch.cc:2523] 'pytorch' TRITONBACKEND API version: 1.15
I1122 15:20:46.868135 167 pinned_memory_manager.cc:241] Pinned memory pool is created at '0xfffe47200000' with size 268435456
I1122 15:20:46.868314 168 pinned_memory_manager.cc:241] Pinned memory pool is created at '0xfffe4ee00000' with size 268435456
I1122 15:20:46.882071 167 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1122 15:20:46.882088 167 cuda_memory_manager.cc:107] CUDA memory pool is created on device 1 with size 67108864
I1122 15:20:46.883199 168 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1122 15:20:46.883219 168 cuda_memory_manager.cc:107] CUDA memory pool is created on device 1 with size 67108864
W1122 15:20:47.218429 168 server.cc:249] failed to enable peer access for some device pairs
W1122 15:20:47.218896 167 server.cc:249] failed to enable peer access for some device pairs
I1122 15:20:47.221528 167 model_lifecycle.cc:462] loading: postprocessing:1
I1122 15:20:47.221537 168 model_lifecycle.cc:462] loading: preprocessing:1
I1122 15:20:47.221584 168 model_lifecycle.cc:462] loading: postprocessing:1
I1122 15:20:47.221588 167 model_lifecycle.cc:462] loading: tensorrt_llm:1
I1122 15:20:47.221629 167 model_lifecycle.cc:462] loading: preprocessing:1
I1122 15:20:47.221630 168 model_lifecycle.cc:462] loading: tensorrt_llm:1
E1122 15:20:47.229456 167 model_lifecycle.cc:622] failed to load 'tensorrt_llm' version 1: Not found: unable to load shared library: /opt/tritonserver/backends/tensorrtllm/libtriton_tensorrtllm.so: undefined symbol: __aarch64_swp4_sync
I1122 15:20:47.229482 167 model_lifecycle.cc:757] failed to load 'tensorrt_llm'
E1122 15:20:47.229518 168 model_lifecycle.cc:622] failed to load 'tensorrt_llm' version 1: Not found: unable to load shared library: /opt/tritonserver/backends/tensorrtllm/libtriton_tensorrtllm.so: undefined symbol: __aarch64_swp4_sync
I1122 15:20:47.229543 168 model_lifecycle.cc:757] failed to load 'tensorrt_llm'
I1122 15:20:47.234780 167 python_be.cc:2115] TRITONBACKEND_ModelInstanceInitialize: postprocessing_0_0 (CPU device 0)
I1122 15:20:47.235271 167 python_be.cc:2115] TRITONBACKEND_ModelInstanceInitialize: preprocessing_0_0 (CPU device 0)
I1122 15:20:47.237517 168 python_be.cc:2115] TRITONBACKEND_ModelInstanceInitialize: preprocessing_0_0 (CPU device 0)
I1122 15:20:47.237673 168 python_be.cc:2115] TRITONBACKEND_ModelInstanceInitialize: postprocessing_0_0 (CPU device 0)
I1122 15:20:47.875521 167 model_lifecycle.cc:819] successfully loaded 'postprocessing'
I1122 15:20:47.875541 168 model_lifecycle.cc:819] successfully loaded 'postprocessing'
I1122 15:20:49.456834 167 model_lifecycle.cc:819] successfully loaded 'preprocessing'
E1122 15:20:49.456889 167 model_repository_manager.cc:563] Invalid argument: ensemble 'ensemble' depends on 'tensorrt_llm' which has no loaded version. Model 'tensorrt_llm' loading failed with error: version 1 is at UNAVAILABLE state: Not found: unable to load shared library: /opt/tritonserver/backends/tensorrtllm/libtriton_tensorrtllm.so: undefined symbol: __aarch64_swp4_sync;
I1122 15:20:49.456970 167 server.cc:604]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+