support eos_token list in turbomind

[irexyc]

Motivation

MinLengthLogitsProcessor in transformers supports list of eos_token_id

Modification

diff --git a/benchmark/profile_throughput.py b/benchmark/profile_throughput.py
index 2e4d2a3b..9fc9605f 100644
--- a/benchmark/profile_throughput.py
+++ b/benchmark/profile_throughput.py
@@ -108,6 +108,7 @@ class Engine:
                 session_id,
                 input_ids=input_ids,
                 gen_config=GenerationConfig(max_new_tokens=output_seqlen,
+                                            min_new_tokens=output_seqlen - 1,
                                             temperature=temperature,
                                             top_p=top_p,
                                             top_k=top_k,

FT_NVTX=ON /mnt/141/2024.5.1/target-linux-x64/nsys profile -t cuda,nvtx,osrt,cudnn,cublas -o output -f true --stats true  python ../benchmark/profile_throughput.py -n 20000 /home/chenxin/ShareGPT_V3_unfiltered_cleaned_split.json /home/chenxin/Llama-3.2-1B-Instruct


 Time (%)  Total Time (ns)  Instances  Avg (ns)  Med (ns)  Min (ns)  Max (ns)  StdDev (ns)                                                  Name                                                
 --------  ---------------  ---------  --------  --------  --------  --------  -----------  ----------------------------------------------------------------------------------------------------
      0.0         45986089      16761    2743.6    2656.0      2272      3840        276.0  void turbomind::batchApplyMinLengthPenalty<float>(T1 *, const int *, const int *, const int *, int,…

(sz=1)0.0         48248303      16797    2872.4    2816.0      2464      3808        237.3  void turbomind::batchApplyMinLengthPenalty<float>(T1 *, int, const int *, const int *, int, const i…
(sz=2)0.0         45838054      16854    2719.7    2656.0      2240      3552        170.6  void turbomind::batchApplyMinLengthPenalty<float>(T1 *, int, const int *, const int *, int, const i…
(sz=4)0.0         45971238      16826    2732.2    2688.0      2304      3584        167.5  void turbomind::batchApplyMinLengthPenalty<float>(T1 *, int, const int *, const int *, int, const i…
(sz=8)0.0         45888590      16819    2728.4    2656.0      2240      3680        171.4  void turbomind::batchApplyMinLengthPenalty<float>(T1 *, int, const int *, const int *, int, const i…

[lvhan028]

Overall LGTM