Cross-val splitter

[jeffjennings]

• Moves datasets.KFoldCrossValidatorGridded to crossval.DartboardSplitGridded, updates imports of this throughout tests and tutorials. I don't like renaming a user-facing function, but I think with this PR's new splitting class (crossval.RandomCellSplitGridded) and new class that performs cross-validation (crossval.CrossValidate), the name KFoldCrossValidatorGridded is confusing (e.g. it's not doing the cross-val, just splitting).

  • moves tests using crossval.DartboardSplitGridded into crossval_test

• Adds another dataset splitting class crossval.RandomCellSplitGridded, which draws random cells from the gridded vis to build the (train, test) sets, returning an iterator of GriddedDataset tuples, consistent with crossval.DartboardSplitGridded.

  • Cells in the top 1% of gridded weight are included in every train set
  • The split sets are even in number:

 for (train, test) in DartboardSplitGridded:
     print(train.shape, test.shape)

              torch.Size([19580]) torch.Size([5798])
              torch.Size([19492]) torch.Size([5886])
              torch.Size([20734]) torch.Size([4644])
              torch.Size([21266]) torch.Size([4112])
              torch.Size([20440]) torch.Size([4938])

 for (train, test) in RandomCellSplitGridded:
     print(train.shape, test.shape)

              torch.Size([20353]) torch.Size([5025])
              torch.Size([20353]) torch.Size([5025])
              torch.Size([20353]) torch.Size([5025])
              torch.Size([20353]) torch.Size([5025])
              torch.Size([20354]) torch.Size([5024])

• Updates crossval.CrossValidate workflow to use these classes to split the data (crossval.CrossValidate.split_dataset), within a k-fold cross-val loop (crossval.CrossValidate.run_crossval)

  • generates one train and one test GriddedDataset at a time in the loop to keep memory footprint reasonable

• Haven't confirmed GPU compatibility yet (will address in Fit runner #128 once I include calls to crossval.CrossValidate in the fit pipeline)

• Haven't added a stratified k-fold splitter - let's see how RandomCellSplitGridded does

[jeffjennings]

Testing whether review request triggers docs_build workflow once or twice...

[jeffjennings]

...just once!

[jeffjennings]

Added note in #100 about changes here.