Question on understanding how model.forward factors in temperature?

[efawe]

Hi,

I had a question about model.forward/inference and temperature.
I know that temperature impacts the softmax scores from logits.
Currently, I want to manually alter the temperature and obtain the softmax scores from the logits outputted by model.forward.

I know that even before calling model.forward, I have to initialize the model. In doing so, I already set the temperature to 0.05.
I don't know if in calling model.forward, the logits are already subject to the 0.05 temperature? If so do I then need to find some proportional operation to change the scaling from 0.05 temperature to 0.5 (for example). Or is this impossible and I need to just rerun the whole model with 0.5 temperature?
Or can I simply just take the logits and create a function that allows me to apply temperature with respect to the softmax.

[turboderp]

model.forward just returns the raw logits. Temperature isn't applied until the softmax step in the sampler.