Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Feb 12, 2025.

This update includes:

• Model Support
  • Added DeepSeek V3/R1 support on PyTorch workflow.
• Features
  • LLM API
    • Added batched logits processor support.
    • Added EAGLE support.
  • Added FP8 quantization support for Qwen2-VL.
  • Added batched inference support for the LLM API MMLU example examples/mmlu_llmapi.py.
  • Added FP4 quantization-layernorm fusion plugin support. (Llama models only)
• API
  • Exposed kc_cache_retention_config from C++ executor API to the LLM API.

Thanks,
The TensorRT-LLM Engineering Team

[akhoroshev]

When will deepseek r1 be available in the trtllm api (not PyTorch)?