why the final model params files can‘t load with transformers class?

[NoobPythoner]

 As the title indicates，this aspect determines whether this framework can be generalized/standardized.
 The implementation of generate_simple is somewhat absurd as it requires manually specifying the length of the generated content each time.

[NoobPythoner]

is that because not generate model.safetensors.index.json ,the map files?

[turboderp]

generate_simple is not really meant to be anything but a quick little test to see if you're getting coherent text from the model.

The streaming generator which inherits from it has a more advanced loop which can generate to an EOS token or stop string, though it currently does not support batching. You can call stream() in a loop and concatenate the output text until it signals that it's done, something like:

response = ""
eos = False
while not eos:
    chunk, eos, _ = generator.stream()
    response += chunk

There are more generator functions coming, of course. Things take time. As for standardizing, I'm not sure what the standard is supposed to be. (?)

And the model.safetensors.index.json isn't produced by the quantizer because it isn't needed for anything. Unlike Pickle files, safetensors files each contain an index that you can just read without having to scan the entire file.

[NoobPythoner]

My requirement is only one, why can't I directly use the Transformers library to load it?

[turboderp]

Because it's not written as a HF Transformers module. It's a different implementation. Just like you can't load GGML/GGUF models, there isn't support for it in Transformers.