Installing litgpt should not downgrade PyTorch

[t-vi]

Bug description

#1796 pins the version to <= 2.4.1, downgrading my PyTorch on pip install. I see that there are compatibility reasons, but to my mind, the cure is a lot worse than the disease. For starters, it will break anyone using anything but the default PyTorch from pip (as in all but one variant from the matrix on the PyTorch get started locally page).