support for T4

[krishnanpooja]

System Info

T4 GPU
TensorRT-LLM: 0.15.0.dev2024111200

Who can help?

@ncomly-nvidia , @juney-nvidia , @byshiue

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Build models for tensorrt-llm engine on T4 GPU.

Expected behavior

Should work on T4 GPU

actual behavior

I am seeing warning message
[12/24/2024-10:21:22] [TRT-LLM] [W] Failed to infer cluster info for Tesla T4, treat it as a L40 node with 15 GB memory. This setting makes no effect if you do not use auto parallel.

How do I get it to recognize T4?

additional notes

The model works on A100. Both single and mGPU setup

[nv-guomingz]

Hi @krishnanpooja , T4 is no longer a support hardware since 0.14 release.