Recovering rescored logits

[OllieBroadhurst]

Hi there!

I was wondering if it would at all be possible to recover the updated model logits after the decoder has rescored them? From what I can tell, the logits themselves are "lost" during decoding so I have my doubts.

The use case is a bit of an edge case, but I'd like to use the updated logits for teacher-student training where the teacher model has benefitted from the LM beforehand.

Thanks!