Update the checkpoint loader logic for inference

Update the checkpoint loader logic for inference #1455