Skip to content

Commit

Permalink
update description
Browse files Browse the repository at this point in the history
Signed-off-by: KumoLiu <[email protected]>
  • Loading branch information
KumoLiu committed Sep 8, 2023
1 parent 0c17b1d commit 543043f
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion acceleration/fast_model_training_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ train_transforms = [
]
dataset = CacheDataset(..., transform=train_trans)
```
Here we convert to PyTorch `Tensor` with `EnsureTyped` transform and move data to GPU with `ToDeviced` transform. `CacheDataset` caches the transform results until `ToDeviced`, so it is in GPU memory. Then in every epoch, the program fetches cached data from GPU memory and only execute the random transform `RandCropByPosNegLabeld` on GPU directly.
Here we convert to PyTorch `Tensor` and move data to GPU with `EnsureTyped` transform. `CacheDataset` caches the transform results until `EnsureTyped`, so it is in GPU memory. Then in every epoch, the program fetches cached data from GPU memory and only execute the random transform `RandCropByPosNegLabeld` on GPU directly.
GPU caching example is available at [Spleen fast training tutorial](fast_training_tutorial.ipynb).

## Leveraging multi-GPU distributed training
Expand Down
2 changes: 1 addition & 1 deletion acceleration/fast_training_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@
"For MONAI fast training progress, we mainly introduce the following features:\n",
"1. `AMP` (auto mixed precision): AMP is an important feature released in PyTorch v1.6, NVIDIA CUDA 11 added strong support for AMP and significantly improved training speed.\n",
"2. `CacheDataset`: Dataset with the cache mechanism that can load data and cache deterministic transforms' result during training.\n",
"3. `ToDeviced` transform: to move data to GPU and cache with `CacheDataset`, then execute random transforms on GPU directly, avoid CPU -> GPU sync in every epoch. Please note that not all the MONAI transforms support GPU operation so far, still working in progress.\n",
"3. `EnsureTyped` transform: to move data to GPU and cache with `CacheDataset`, then execute random transforms on GPU directly, avoid CPU -> GPU sync in every epoch. Please note that not all the MONAI transforms support GPU operation so far, still working in progress.\n",
"4. `set_track_meta(False)`: to disable meta tracking in the random transforms to avoid unnecessary computation.\n",
"5. `ThreadDataLoader`: uses multi-threads instead of multi-processing, faster than `DataLoader` in light-weight task as we already cached the results of most computation.\n",
"6. `DiceCE` loss function: computes Dice loss and Cross Entropy Loss, returns the weighted sum of these two losses.\n",
Expand Down

0 comments on commit 543043f

Please sign in to comment.