studio: push exp refs

[dberenbaum]

DVCLive should dvc exp push experiment refs when:

1. STUDIO_TOKEN is set (or live sharing to Studio is otherwise enabled)
2. save_dvc_exp=True (even if using dvc exp run; not sure what's best if using dvc exp run but not save_dvc_exp=True)

On the Studio side, this dvc exp push should replace the live metrics with the exp ref. This would help unify the live metrics and exp refs. For typical workflows, it means that live metrics would show live updates only temporarily and eventually show a "completed" exp ref.

Edit: update from related discussion. DVCLive should not have to push experiment refs. DVC should push them during either dvc exp save or dvc exp run if the following are both true:

1. The Studio token is available. This can be found either in the studio.token config option or the DVC_STUDIO_TOKEN env var (let's rename STUDIO_TOKEN to DVC_STUDIO_TOKEN for consistency with other env vars). In the future, we could also have a way to pass the token via CLI/Python API. We can follow the same conventions for the Studio URL.
2. Automatic sharing/pushing is enabled. This can be enabled via the exp.auto_push config option or the DVC_EXP_AUTO_PUSH env var. Like the Studio token, we could also have a way to enable/disable through CLI/Python API. We can follow the same conventions for the Git remote URL, which should be optional.

This means including STUDIO_TOKEN/DVC_STUDIO_TOKEN would not be enough on its own to enable live sharing of experiments. I think it's okay because I would push for the onboarding to be made easier via #9265 to not rely on this env var and instead have the env var mostly used for CI.

[dberenbaum]

I think this could actually be moved to a DVC issue. It should be that every DVC exp commit generated by dvc exp run/save should get pushed when live sharing is enabled. Related to #9074 (comment).

This also probably requires inferring a default Git branch to push to (there is a related discussion about dvc exp push that I can't seem to find).

[daavoo]

I think this could actually be moved to a DVC issue. It should be that every DVC exp commit generated by dvc exp run/save should get pushed when live sharing is enabled. Related to #9074 (comment).

It makes sense to me.

This also probably requires inferring a default Git branch to push to

Not sure I understand what is that requirement about

[dberenbaum]

Sorry, I meant inferring a default Git remote.

[daavoo]

Sorry, I meant inferring a default Git remote.

It appears that it is already handled in the current exp push:

dvc/dvc/repo/experiments/push.py

Line 59 in 24e10e0

_, repo_url = get_remote_repo(repo.scm.dulwich.repo, git_remote)

[dberenbaum]

🤨 Then why does dvc exp push require a git remote argument and auto-pushing requires DVC_EXP_GIT_REMOTE?

[daavoo]

🤨 Then why does dvc exp push require a git remote argument and auto-pushing requires DVC_EXP_GIT_REMOTE?

Because those 2 things (remote arg, env var) are from before the code I linked, which was introduced for Studio integration.
More discussion here https://iterativeai.slack.com/archives/CNQ95CG1K/p1679983222971979

[dberenbaum]

I'm in favor of making the git remote optional in all these places. Even in a forking workflow, I don't really see the problem since there's an option to configure it, and the typical workflow would still be to push to origin.

I know @pmrowla suggested he typically pushes to upstream, but I can't recall the exact workflow there. It seems there is otherwise consensus that this is more burdensome than helpful.

[skshetry]

It appears that it is already handled in the current exp push:

dvc/dvc/repo/experiments/push.py

Line 59 in 24e10e0

_, repo_url = get_remote_repo(repo.scm.dulwich.repo, git_remote)

It's not inferring default remote, it's using the same remote where dvc exp push pushed, unlike what we do in dvc-studio-client.

[pmrowla]

IIRC DVC_EXP_GIT_REMOTE was something that had been requested on the CML side, so that exp's run inside CI workflows would be preserved, but I'm not sure if anyone is actually using it