Workflow Pipeline Orchestration Template for Pathology Tasks

[cpoptic]

With all the available workflow pipeline orchestration tools (e.g. Prefect, ZenML, Flyte, KubeFlow Pipelines, etc), there are many options to build an end-to-end pipeline for a given histopathology task.

I think many/most users would find it very helpful to have a standardized pipeline template, such that each step or component module of that pipeline is loosely coupled, with strong interfaces and can be modified easily without impacting the immediate upstream and downstream tasks in the pipeline. Each component would thus be seen as an abstraction where data flows into and out of each component module.

For example, a Whole Slide Image (WSI) classification task pipeline shares some components regardless of the specific dataset being used: There is probably a need for:

• a Data Ingestion component
• a Data Quality component (to check for and exclude WSI tissue anomalies, pen-marks, etc)
  ** e.g. using a tool to validation vision data like DeepChecks
• a Data Splitting component (training, validation, test split)
• a Patch Generator component (that divides a huge Giga-pixel WSI into smaller tiles/patches)
• a Model Training component
• a Patch Aggregation component (to aggregate the individual patch-level predictions into a whole slide-level prediction)
• an Evaluator component that runs inference on the test dataset and calculates evaluation metrics (AUC, precision, recall, etc)
• ...etc

This Workflow Pipeline should be flexible enough such that the user can tweak the details of an individual component (e.g. perhaps they want to change the way that patches are applied to a WSI, or perhaps they want to use a Multiple Instance Learning classifier model vs an Attention-Based MIL model, etc).

The idea is that some structure would help significantly in building out these pipelines.

This is probably related to issue #4860

In a related issue, are there plans to integrate MONAI with a configuration management tool like Hydra, with PyTorch Lightning, and with a Workflow orchestration tool like Flyte or ZenML?

I think these integrations would add tremendous value to what MONAI already has wonderfully built thus far.

Some literature regarding this important issue:

• "Built to last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology" (Wagner, 2022)
• "Building tools for machine learning and artificial intelligence in cancer research- best practices and a case study with the PathML toolkit for computational pathology" (Rosenthal, 2021)
• "Reproducibility standards for machine learning in the life sciences" (Heil, 2021)

[wyli]

In a related issue, are there plans to integrate MONAI with a configuration management tool like Hydra, with PyTorch Lightning

for this bit we've implemented a light-weight config parser for json/yaml/hybrid programming (https://github.com/Project-MONAI/tutorials/tree/main/modules/bundle), and currently we mainly integrated pytorch-ignite workflow engine https://github.com/Project-MONAI/MONAI/blob/dev/monai/engines/workflow.py