Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PythonicWorkflow #8151

Merged
merged 44 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
44752e6
Removed CPU randn() from schedulers
borisfom Oct 12, 2024
1bf3edd
Merge branch 'dev' into fix-schedulers
borisfom Oct 14, 2024
df213e6
Merge remote-tracking branch 'origin/dev' into fix-schedulers
borisfom Oct 14, 2024
f5fa56b
Merge branch 'fix-schedulers' of github.com:borisfom/MONAI into fix-s…
borisfom Oct 14, 2024
1893375
workaround for #8149
KumoLiu Oct 15, 2024
1c126a9
add `PythonicWorkflow`
KumoLiu Oct 15, 2024
f431f6c
Merge remote-tracking branch 'origin/dev' into pythonicworkflow
KumoLiu Oct 15, 2024
7228989
minor update
KumoLiu Oct 15, 2024
cf31a38
support for multipy properties file
KumoLiu Oct 17, 2024
b94b2a0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 17, 2024
7248c2b
minor fix
KumoLiu Oct 18, 2024
800dd4e
Merge branch 'pythonicworkflow' of https://github.com/KumoLiu/MONAI i…
KumoLiu Oct 18, 2024
17da2ae
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 7, 2024
260cb63
Merge remote-tracking branch 'yunl/pythonicworkflow' into pythonicwor…
KumoLiu Nov 7, 2024
1b39551
Merge remote-tracking branch 'origin/dev' into pythonicworkflow
KumoLiu Nov 11, 2024
f99763d
add test case
KumoLiu Nov 11, 2024
6441466
fix format
KumoLiu Nov 11, 2024
c2e2dad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 11, 2024
17e4f32
add docstring
KumoLiu Nov 11, 2024
d1678b0
Merge branch 'pythonicworkflow' of https://github.com/KumoLiu/MONAI i…
KumoLiu Nov 11, 2024
6b2817e
fix format
KumoLiu Nov 11, 2024
6eed10d
update docstring
KumoLiu Nov 11, 2024
f31e39d
fix mypy
KumoLiu Nov 11, 2024
7f5c43b
Update monai/bundle/workflows.py
KumoLiu Nov 13, 2024
9c3155f
address comments
KumoLiu Nov 13, 2024
ef41603
fix format
KumoLiu Nov 13, 2024
891b9e3
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 13, 2024
4857445
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 14, 2024
9acad6b
fix ci
KumoLiu Nov 14, 2024
78d6da7
Merge remote-tracking branch 'yunl/pythonicworkflow' into pythonicwor…
KumoLiu Nov 14, 2024
99af7fd
address comments
KumoLiu Nov 14, 2024
f933263
address comments
KumoLiu Nov 15, 2024
4724f68
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 15, 2024
50b1a50
address comments
KumoLiu Nov 15, 2024
150a60a
address comments
KumoLiu Nov 15, 2024
b0acb17
Merge branch 'pythonicworkflow' of https://github.com/KumoLiu/MONAI i…
KumoLiu Nov 15, 2024
c026441
fix format
KumoLiu Nov 15, 2024
5ffc03d
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 15, 2024
9f9ebcd
Merge remote-tracking branch 'yunl/pythonicworkflow' into pythonicwor…
KumoLiu Nov 15, 2024
ec202e0
update `compute_capabilities_after`
KumoLiu Nov 15, 2024
70dc9b5
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 25, 2024
d40ec95
Merge branch 'dev' into pythonicworkflow
KumoLiu Nov 27, 2024
42d5d0b
fix docstring
KumoLiu Nov 27, 2024
1f136f9
Merge branch 'pythonicworkflow' of https://github.com/KumoLiu/MONAI i…
KumoLiu Nov 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion monai/bundle/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,4 @@
MACRO_KEY,
load_bundle_config,
)
from .workflows import BundleWorkflow, ConfigWorkflow
from .workflows import BundleWorkflow, ConfigWorkflow, PythonicWorkflow
109 changes: 109 additions & 0 deletions monai/bundle/workflows.py
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,115 @@ def check_properties(self) -> list[str] | None:
return [n for n, p in self.properties.items() if p.get(BundleProperty.REQUIRED, False) and not hasattr(self, n)]


class PythonicWorkflow(BundleWorkflow):
"""
Base class for the pythonic workflow specification in bundle, it can be a training, evaluation or inference workflow.
It defines the basic interfaces for the bundle workflow behavior: `initialize`, `run`, `finalize`, etc.
And also provides the interface to get / set public properties to interact with a bundle workflow.
KumoLiu marked this conversation as resolved.
Show resolved Hide resolved

Args:
workflow_type: specifies the workflow type: "train" or "training" for a training workflow,
yiheng-wang-nv marked this conversation as resolved.
Show resolved Hide resolved
or "infer", "inference", "eval", "evaluation" for a inference workflow,
other unsupported string will raise a ValueError.
default to `train` for train workflow.
workflow: specifies the workflow type: "train" or "training" for a training workflow,
or "infer", "inference", "eval", "evaluation" for a inference workflow,
other unsupported string will raise a ValueError.
default to `None` for common workflow.
properties_path: the path to the JSON file of properties.
meta_file: filepath of the metadata file, if this is a list of file paths, their contents will be merged in order.
logging_file: config file for `logging` module in the program. for more details:
https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig.

"""

supported_train_type: tuple = ("train", "training")
supported_infer_type: tuple = ("infer", "inference", "eval", "evaluation")

@deprecated_arg(
"workflow",
since="1.2",
removed="1.5",
new_name="workflow_type",
msg_suffix="please use `workflow_type` instead.",
)
ericspod marked this conversation as resolved.
Show resolved Hide resolved
def __init__(
self,
workflow_type: str | None = None,
workflow: str | None = None,
properties_path: PathLike | None = None,
meta_file: str | Sequence[str] | None = None,
logging_file: str | None = None,
):
super().__init__(workflow_type=workflow_type, workflow=workflow, properties_path=properties_path, meta_file=meta_file, logging_file=logging_file)
self._props = {}
self._set_props = {}

def initialize(self, *args: Any, **kwargs: Any) -> Any:
"""
Initialize the bundle workflow before running.
"""
self._props = {}

@abstractmethod
def run(self, *args: Any, **kwargs: Any) -> Any:
"""
Run the bundle workflow, it can be a training, evaluation or inference.

"""
raise NotImplementedError()

@abstractmethod
def finalize(self, *args: Any, **kwargs: Any) -> Any:
"""
Finalize step after the running of bundle workflow.

"""
raise NotImplementedError()
KumoLiu marked this conversation as resolved.
Show resolved Hide resolved

def _get_property(self, name: str, property: dict) -> Any:
"""
With specified property name and information, get the expected property value.
If the property is already generated, return from the bucket directly.
If user explicitly set the property, return it directly.
Otherwise, generate the expected property as a class private property with prefix "_".

Args:
name: the name of target property.
property: other information for the target property, defined in `TrainProperties` or `InferProperties`.
"""
value = None
if name in self._set_props:
value = self._set_props[name]
self._props[name] = value
KumoLiu marked this conversation as resolved.
Show resolved Hide resolved
elif name in self._props:
value = self._props[name]
else:
try:
value = getattr(self, f"get_{name}")()
except AttributeError:
if property[BundleProperty.REQUIRED]:
raise ValueError(
f"unsupported property '{name}' is required in the bundle properties,"
f"need to implement a method 'get_{name}' to provide the property."
)
self._props[name] = value
return value

def _set_property(self, name: str, property: dict, value: Any) -> Any:
"""
With specified property name and information, set value for the expected property.
Stores user-reset initialized objects that should not be re-initialized.

Args:
name: the name of target property.
property: other information for the target property, defined in `TrainProperties` or `InferProperties`.
value: value to set for the property.

"""
self._set_props[name] = value


binliunls marked this conversation as resolved.
Show resolved Hide resolved
class ConfigWorkflow(BundleWorkflow):
"""
Specification for the config-based bundle workflow.
Expand Down
Loading