ChatGLM2-6B tokenizer can't load?

[THU-mjx]

when i built a ChatGLM2-6B engine to launch the triton-server, an error occurred: Tokenizer class ChatGLMTokenizer does not exist

[byshiue]

Hope this issue is helpful THUDM/ChatGLM2-6B#243.

[THU-mjx]

when i add "trust_remode_code=True" in preprocessing/&postprocessing/1/model.py，
another error occurred : Failed to initialize Python stub: AttributeError: can‘t set attribute ’pad_token

I tried to fix it with this methods

The server could be built, but can't be used!

[byshiue]

What branch do you use? If you don't use the latest main branch, please try it. Please remember updating the tensorrt_llm, too. Also, you need to rebuild the tensorrt_llm and the engine when you update the codes.

[THU-mjx]

Using the latest tensorrt_llm and tensorrtllm_backend, I failed to launch the server with the above error:

[THU-mjx]

I fixed this issue. The latest version of trtllm-backend have to manually set max_batch_size.
I have another question : how to set these values?

[byshiue]

You can set 1 first.

[THU-mjx]

Another error happened in preprocessing/postprocessing/model.py （I test several models, the errors are same!）

IN model.py, i just added 'trust_remote_code=True'

[byshiue]

Please refer THUDM/ChatGLM3#152.

A possible workaround is preventing assign the pad_token, and replace all pad_token by eos_token.