Skip to content

Commit

Permalink
add timing callback (#657)
Browse files Browse the repository at this point in the history
### Description
Add timing callback to log train step time in second

- [ ]  Bug fix (non-breaking change which fixes an issue)
- [x]  New feature (non-breaking change which adds functionality)
- [ ]  Refactor
- [ ]  Documentation update
- [ ]  Other (please describe):

Signed-off-by: sichu <[email protected]>
  • Loading branch information
sichu2023 authored Jan 26, 2025
1 parent b589fc6 commit 5d40adb
Showing 1 changed file with 2 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
from nemo.lightning import resume
from nemo.lightning.pytorch import callbacks as nl_callbacks
from nemo.lightning.pytorch.optim import MegatronOptimizerModule
from nemo.utils.exp_manager import TimingCallback

from bionemo.core.utils.dtypes import PrecisionTypes, get_autocast_dtype
from bionemo.esm2.api import ESM2Config
Expand Down Expand Up @@ -217,6 +218,7 @@ def main(
RichModelSummary(max_depth=4),
LearningRateMonitor(),
nl_callbacks.PreemptionCallback(),
TimingCallback(),
]
if nsys_profiling:
if nsys_end_step is None:
Expand Down

0 comments on commit 5d40adb

Please sign in to comment.