next_step() needs log()

[casperdcl]

import dvclive

live = dvclive.Live(resume=True)
while (epoch := live.get_step()) < 10:
  print(epoch)
  live.log("epoch", epoch)  # need to log at least one thing for `next_step` to work
  live.next_step()

• current: dvclive.json is stuck on step: 0 if we remove the live.log() command.
• current (after live: Moved all make_{x} to next_step(). #353) dvclive.json doesn't exists if we remove the live.log() command.
• expected: step: 9 even without live.log()

Related: iterative/cml.dev#207

[daavoo]

The step is being correctly updated internally (even if you remove the log call, the while loop breaks) but the updates are not being dumped to the summary file dvclie.json. This is because we moved the summary updates to log calls (#145).

Not sure what to do here. The summary is intended to track logged scalars and we just include step for convenience 🤔

cc @dberenbaum

[casperdcl]

Seems odd to do this. If there are multiple things to log per iteration, you dump multiple times?

• Seems better to dump only on next_step()
• Could also make configurable default log(..., write_summary=False)
• Could make next_step() internally use log("step", ..., write_summary=True)

I don't quite follow the wording in #145 but from what I can tell it seems identical to this issue (#232) so shouldn't have been closed?

[dberenbaum]

@casperdcl Why do you want to use next_step() without log()? I don't see a problem doing so, but I'm confused how it's useful.

Edit: To clarify, I meant that your expectations seem reasonable, but I'd like to better understand the context.

[casperdcl]

Full context is step is the only thing that is required in order to resume interrupted training (notice the example above uses get_step()) - we don't need to log loss nor any other metrics, and we don't need to use dvc {exp run,repro}.

Second unrelated lower-priority follow-up issue is optimisation:

If there are multiple things to log per iteration, you dump multiple times?

[dberenbaum]

We probably need to decouple log() from setting the step to resolve this issue.

@casperdcl I moved the optimization to #238.

Both are good points that are largely due to changes in the codebase over time and should probably be changed now AFAIK.

[dberenbaum]

Actually, I think we just need to call make_summary() always during set_step() (ignoring #238).