Questions about lazy transforms

[function2-llx]

Dear MONAI team and community,

I recently tried the feature of lazy transform, and it's amazing, thank you for your work on this! Currently, I've got some minor questions.
The first question is about lazy resample. From my understanding, when applying pending transforms, transform affine matrix will be accumulated and a spatial resample will be performed at last. The implementation of SpatialResample solves the transform affine from src_affine to dst_affine:

MONAI/monai/transforms/spatial/functional.py

Line 128 in 17c1e3a

xform = np.eye(spatial_rank + 1) if spatial_rank < 2 else np.linalg.solve(_s, _d)

This seems a little redundant (and may very slightly lose some numerical precision) since dst_affine is obtained by multiplying the initial affine matrix (i.e., src_affine) and the accumulated transform affine matrix:

MONAI/monai/transforms/lazy/utils.py

Line 191 in 17c1e3a

"dst_affine": init_affine @ monai.utils.convert_to_dst_type(matrix, init_affine)[0],

Well, after all, the current implementation works pretty fine, I'm just curious about this.

The second question (or maybe a feature request) is about the compose transform. It seems that the compose transform always applies all pending transforms at last:

MONAI/monai/transforms/compose.py

Line 114 in 17c1e3a

data = apply_pending_transforms(data, None, overrides, logger_name=log_stats)

However, I feel that sometimes I would like it to be optional, e.g., when performing some intermediate transforms, I use Compose to construct a series of lazy transforms, and I don't want the pending transforms to be applied at once.

Any comments will be appreciated.

Best wishes

[KumoLiu]

Hi @function2-llx, thanks for your interest here!

• For the question, since we only need to compute the affine change once, this operation should reduce the loss of precision. It also reduces the loss of image detail due to multiple spatial transforms.
• For now, we only support executing the transforms in order, i.e. if a non-lazy transform is encountered, then all the pending will be applied. Hopefully, we will support more advanced modes to apply the transforms in the near future.

FYI, we also have several tutorials that you can refer to.
https://github.com/Project-MONAI/tutorials/blob/main/modules/lazy_resampling_benchmark.ipynb
https://github.com/Project-MONAI/tutorials/blob/main/modules/lazy_resampling_compose.ipynb
https://github.com/Project-MONAI/tutorials/blob/main/modules/lazy_resampling_functional.ipynb

Thanks!

[wyli]

yes, the src/dst and transform conversion is mainly to make the API generic, it's also used here

MONAI/monai/data/image_writer.py

Line 208 in 17c1e3a

def resample_if_needed(

and indirectly here

MONAI/monai/data/itk_torch_bridge.py

Line 68 in 17c1e3a

def metatensor_to_itk_image(

[atbenmurray]

Hi @function2-llx, just seen this referenced from #7139. I'll address each point in turn:

• Pt1: I think I agree, but I'd also like to understand your concern better
• Pt2: We did have some out of order execution modes slated for 1.2 but these were dropped as it was felt that it is too complex for the majority of users and could easily cause unexpected outputs