LayerNorm_A calculation precision is unstable, leading to incorrect recognition results.

[winf]

#41 #20
When using models other than .en, if the model size exceeds ‘tiny’, the recognition results are found to be significantly incorrect. By comparing the layer-by-layer outcomes with the openai repository’s runtime results, it was ultimately determined that the source of the problem is the poor stability of mean and variance calculations in the LayerNorm_A function. I fixed the issue by replacing fp16 with fp32. Reference: https://git.bwbot.org/publish/useful-transformers/-/blob/main/lib/layernorm.cc

[winf]

@marty1885 @guynich

[marty1885]

Thank you! This solves my problems too!

I don't have write access to the repository. Maybe you could make a PR and upstream the changes? I can test on my end.