Skip to content

Commit

Permalink
cli/benchmark: fix null-div of first epoch
Browse files Browse the repository at this point in the history
Co-authored-by: Julien Vignoud <[email protected]>
  • Loading branch information
tharvik and JulienVig authored Jul 2, 2024
1 parent a934076 commit 5a2f781
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion cli/src/benchmark_gpt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ async function main(args: Required<CLIArguments>): Promise<void> {
console.log(`\tmodel type ${modelType} \n\tbatch size ${batchSize} \n\tcontext length ${contextLength}`)

let epochTime = performance.now()
for (let epochsCounter = 0; epochsCounter < epochsCount; epochsCounter++) {
for (let epochsCounter = 1; epochsCounter <= epochsCount; epochsCounter++) {
const [_, logs] = await async_iterator.gather(model.train(preprocessedDataset))
epochTime = (performance.now() - epochTime)
const msPerToken = epochTime / (batchSize * contextLength * iterationsPerEpoch * epochsCounter)
Expand Down

0 comments on commit 5a2f781

Please sign in to comment.