feat: improve Trainer and DeeprankDataset logic for production testing

[gcroci2]

Main changes:

• Now DeeprankDataset takes as input train_data, that before was called dataset_train. Now train_data can be a DeeprankDataset representing the training set (as before), or a pre-trained model (new feature). It needs to be set only if train is False (as before, so only in validation/testing sets cases). Now we are able to use a test dataset without the need for the original model's training dataset. We can use the info stored in the pre-trained model to inherit the needed attributes.
  • There were many issues with loading the lambda transformations from the torch-saved pre-trained model (testing only use case). I ended up converting the lambdas to strings first, and then saving them. When the pre-trained model is loaded in the GraphDataset class, the strings representing the lambdas are evaluated and converted back to functions.

Secondary changes:

• In DeeprankDataset classes, if target attribute is present (e.g., binary, inherited) but it's not in the HDF5 and we're not in the training phase, now no error is raised. Indeed, it should be possible to run a pre-trained model on data point/s even if the target value/s are not present, for doing predictions only. It's actually a typical test-case scenario, in which we don't have any labels for the new data points that we want to evaluate.
• On the other hand, if we're in the training phase (train = True) and no target is set, or the set target is not in the hdf5 file/s, then a ValueError is raised.
• self.pretrained_model_path is now defined in the init of the Trainer class and defaulted to pretrained_model.
• self.model_load_state_dict is also defined in the init of the Trainer class and defaulted to None. It is assigned to a value only in the case of a pre-trained model or at the end of the training phase. This way in the test() method we can first verify if the model has actually been loaded (pre-trained case) or trained. If not, the test() method now throws an error.
• In the Trainer class' init, before loading parameters and the pre-trained model there was a check for the target, which in the pre-trained model case could be not present at all (it is saved in the model itself, no need to define it in the Trainer instance). I removed the check.
• I added the following attributes in _init_from_dataset, which need to be saved in the model's file for those cases in which we want to test it on some other data without redefining the training set: features_transform, means, devs, target_transform, classes, classes_to_index. I also added them to the model's state dict which is saved at the end of the training (_save_model, same for _load_params).
• Now the saved model contains a key called data_type, needed for checking which type of dataset was used during the training of the model.
• I removed the warning about not having a validation set during the training because it was given at each epoch. Now it's printed only once when you call the train() method.
• When you call torch.load() is called on a model's file which contains GPU tensors, those tensors will be loaded to GPU by default. But if no Cuda was available, the code crashed. Now there is a check for that, and in case Cuda is not available the tensors are loaded into the CPU.
• target_filter wasn't really working. Some other edits made me notice that the functionality was broken, and it's fixed now.

Still to solve:

• For some reason, PyTorch gives a weird error about the Adam optimizer, but only in the 3.11 Python version. I tried to fix the relevant torch packages versions, but it's still failing. I haven't touched anything about the optimizer though. Any idea? @DaniBodor

[DaniBodor]

I unsubscribed to notifications for this PR for now. Please tag me again if needed and/or when you want me to re-review.

[DaniBodor]

Just leaving these as comments for now. Once you/we figure out why the build is failing, I will review that before approving.

[DaniBodor]

It looks to me like the problem with the 3.11 build is really a core change in pytorch. I don't think it'll be easy for us (def not me) to figure out what the problem is. Maybe it's best to create an issue on pytorch and see if they know how to solve.

[github-actions]

This PR is stale because it has been open for 14 days with no activity.

[gcroci2]

I am merging this PR. The issue with Python 3.11 will be solved in another PR.