Transform that deletes items from a dataset

[lukepolson]

Suppose I have the following dataset:

items = [{"data": 4}, {"data": 9}, {"data": 3}, {"data": 7}, {"data": 1}, {"data": 2}, {"data": 5}]

Furthermore, suppose I have a Compose pipeline that applies operations to the data. For example, suppose the first operation is to square all the elements of data.

transform= Compose([SquareValuesd(keys='data')])
items_new = transform(items)

And items_new would look like

[{"data": 16}, {"data": 81}, {"data": 9}, {"data": 49}, {"data": 1}, {"data": 4}, {"data": 25}]

What I would like to append to this pipeline is a transformation that deletes elements from items subject to some condition (where this condition is obtained from a previous operation in the pipeline). For example, suppose I wish to square all the data values with key 'data', then remove all the corresponding elements of the dataset that have squared value greater than 60. My dataset would then look like

items_new = [{"data": 16}, {"data": 9}, {"data": 49}, {"data": 1}, {"data": 4}, {"data": 25}]

Implementation of such a transform might look something like:

def greater_than_60(x):
    return x>60

transform= Compose([SquareValues(keys='data'), DeleteItemsd(keys='data', condition=greater_than_60)])
items_new= transform(items)

Is a function like the DeleteItemsd that I specified above implemented in MONAI? And if not, this seems like an essential feature. For example, if I have a dataset CT scans of patients with varying tumour sizes, and I want to create a dataset where I filter by scans only containing tumours of specific sizes, such a feature would be massively helpful. I know that there are other ways of doing this, but having it consolidated under a single transform would be clean.

[KumoLiu]

Hi @lukepolson,
Thank you for your suggestions! Data filtering is usually done before generating the dataset, dataset only used to preprocessed your retained data. Deleting data while generating a dataset can cause problems with the data length.