Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

316 Open 1,788 Closed
316 Open 1,788 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Error with LoRA Weights Data Type in Quantized TensorRT-LLM Model Execution
#2628 opened Dec 25, 2024 by Alireza3242
[Performance] why do compiled trtllm models have bad performance compared to torch.compile models?
#2627 opened Dec 25, 2024 by FPTMMC
3
[bug] forwardAsync assertion failed: Unable to get batch slot for reqId
#2626 opened Dec 25, 2024 by akhoroshev
Does trtllm-serve support toolparser and guided-decoding? Any plan?
#2624 opened Dec 25, 2024 by dwq370
Error during TensorRT-LLM build - Invalid shape and type mismatch in elementwise addition
#2622 opened Dec 24, 2024 by cocovoc
decoder_input_ids must be set to 2 when using fairseq model
#2621 opened Dec 24, 2024 by cocovoc
support for T4 triaged Issue has been triaged by maintainers
#2620 opened Dec 24, 2024 by krishnanpooja
4 tasks
1
SIGABRT while trying to build trtllm engine for biomistral model on T4 triaged Issue has been triaged by maintainers
#2619 opened Dec 24, 2024 by krishnanpooja
2 of 4 tasks
1
[Performance] What is the purpose of compiling a model？ triaged Issue has been triaged by maintainers
#2617 opened Dec 24, 2024 by Flynn-Zh
4 tasks
1
Phi4 support? triaged Issue has been triaged by maintainers
#2616 opened Dec 24, 2024 by oscarbg
1
gather_generation_logits doesn't seem to work correctly for SequenceClassification models Generic Runtime Investigating triaged Issue has been triaged by maintainers
#2615 opened Dec 24, 2024 by TriLoo
Performance of streaming requests is worse than non-streaming bug Something isn't working Investigating Performance Issue about performance number triaged Issue has been triaged by maintainers
#2613 opened Dec 24, 2024 by activezhao
2 of 4 tasks
How to suppress the WARNING logging？
#2610 opened Dec 24, 2024 by lxp3
Adding custom sampling config triaged Issue has been triaged by maintainers
#2609 opened Dec 23, 2024 by buddhapuneeth
1 of 4 tasks
1
Does prefix caching support VLM models?
#2608 opened Dec 23, 2024 by sleepwalker2017
Gemma 2 LoRA support Investigating Lora/P-tuning triaged Issue has been triaged by maintainers
#2606 opened Dec 21, 2024 by Aquasar11
@byshiue
[Feature Request] Better support for w4a8 quantization Investigating Low Precision Issue about lower bit quantization, including int8, int4, fp8 triaged Issue has been triaged by maintainers
#2605 opened Dec 20, 2024 by ShuaiShao93
@Barry-Delaney
3
SmoothQuant doesn't work with lora bug Something isn't working Investigating Lora/P-tuning triaged Issue has been triaged by maintainers
#2604 opened Dec 20, 2024 by ShuaiShao93
4 tasks
@byshiue
lora doesn't work with --use_fp8_rowwise bug Something isn't working
#2603 opened Dec 20, 2024 by ShuaiShao93
4 tasks
--use_fp8 doesn't work with llama 3.1 8b bug Something isn't working
#2602 opened Dec 20, 2024 by ShuaiShao93
4 tasks
No module named 'tensorrt_llm.bindings' bug Something isn't working
#2599 opened Dec 20, 2024 by WGS-note
2 of 4 tasks
1
[Performance] TTFT of qwen2.5 0.5B model bug Something isn't working
#2598 opened Dec 20, 2024 by ReginaZh
4 tasks
Unable to install TensorRT-LLM
#2597 opened Dec 20, 2024 by gowthamtupili
1
Chinese decoding garbled characters in stream mode
#2595 opened Dec 19, 2024 by fan-niu
trtllm-serve : Failure to launch openAI-API on multiple nodes with 8 GPUs each
#2594 opened Dec 19, 2024 by sivabreddy
1
ProTip! Type g i on any issue or pull request to go back to the issue listing page.