Adds lightning fabric integration

[dberenbaum]

Closes #742. Fabric is Pytorch Lightning's attempt to abstract out the lightweight parts of their codebase so users can benefit from those without having to entirely adapt Lightning.

• Example notebook is included: https://github.com/iterative/dvclive/blob/38d1f40843be5adc49ed718884f04496a7f376ff/examples/DVCLive-Fabric.ipynb
• Docs PR: dvclive: add fabric integration dvc.org#5039

The existing Lightning logger was updated to inherit from this one. To make everything work with the existing Lightning logger, I added a sync method that decouples next_step from saving all the info to Studio, dvc.yaml, etc. I made it public, but no strong opinion on whether it should be public or documented.

See the Tensorboard logger in lightning for an example of another logger:

• Docs
• Fabric logger
• Lightning logger

[codecov-commenter]

Codecov Report

Attention: 130 lines in your changes are missing coverage. Please review.

Comparison is base (8406920) 88.79% compared to head (2f81b3f) 86.81%.

Files	Patch %	Lines
src/dvclive/fabric.py	6.75%	69 Missing ⚠️
tests/frameworks/test_fabric.py	25.00%	27 Missing ⚠️
tests/frameworks/test_lightning.py	0.00%	17 Missing ⚠️
src/dvclive/lightning.py	0.00%	16 Missing ⚠️
src/dvclive/utils.py	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #749      +/-   ##
==========================================
- Coverage   88.79%   86.81%   -1.99%     
==========================================
  Files          53       55       +2     
  Lines        3357     3450      +93     
  Branches      294      304      +10     
==========================================
+ Hits         2981     2995      +14     
- Misses        337      416      +79     
  Partials       39       39              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shcheklein]

seems to be unused?

[shcheklein]

review: ipywidgets was required for me by tqdm to run this cell

[shcheklein]

Great work! Thanks, @dberenbaum for pushing forward integrations. Some comments as I'm reviewing / running it:

• While running and after an experiment I can't see plots in VS Code (while it's visible in Jupyter just fine)
• Review mypy CI errors

[shcheklein]

@dberenbaum looks good to me overall (I didn't review really deep the Fabric existing loggers to actually deeply review the implementation, but overall it looks fine). I've enabled mypy back + left a comment about VS Code to review, probably also Studio needs to be reviewed / tried out.

[dberenbaum]

Should mypy be included in https://github.com/iterative/dvclive/blob/main/.pre-commit-config.yaml?

[shcheklein]

I would ask @skshetry and the team on what the current state of the art for this :)

[dberenbaum]

@skshetry How should mypy checks be handled?

[dberenbaum]

ping @skshetry

[dberenbaum]

I'm able to see it in VS Code and Studio:

Screen.Recording.2023-12-14.at.1.23.08.PM.mov

Screen.Recording.2023-12-14.at.1.22.36.PM.mov

[skshetry]

I would ask @skshetry and the team on what the current state of the art for this :)

We use nox for environment management on py-template based projects. So, you can pip install nox and then nox -s lint will take care of everything related to linting.

Since dvclive seems to have removed nox, you can try adding mypy to pre-commit but up to you.
Look into dvc's pre-commit config.

[dberenbaum]

Depends on #757 now. The remaining test failure is unrelated.