From 5fc7d2316537652c5a8c1451dbea373d5489b06e Mon Sep 17 00:00:00 2001 From: Dave Berenbaum Date: Tue, 18 Jul 2023 17:44:49 -0400 Subject: [PATCH] exp run: add options from repro (#4704) * exp run: add options from repro * drop repro options mention in exp run --- content/docs/command-reference/exp/run.md | 64 ++++++++++++++++++++--- content/docs/command-reference/repro.md | 6 +-- 2 files changed, 61 insertions(+), 9 deletions(-) diff --git a/content/docs/command-reference/exp/run.md b/content/docs/command-reference/exp/run.md index 935452a993..93967c5e38 100644 --- a/content/docs/command-reference/exp/run.md +++ b/content/docs/command-reference/exp/run.md @@ -15,12 +15,15 @@ When called with no arguments, this is equivalent to `dvc repro` followed by ## Synopsis ```usage -usage: dvc exp run [-h] [-q | -v] [-f] - { repro options ... } [-n ] - [-S [:]] +usage: dvc exp run [-h] [-q | -v] [-f] [-i] + [-s] [-p] [-P] [-R] + [-n ] [-S [:]] [--queue] [--run-all] [-j ] [--temp] [-r ] [-C ] [-m ] + [--downstream] [--force-downstream] + [--pull] [--dry] [--allow-missing] + [-k] [--ignore-errors] [targets [targets ...]] positional arguments: @@ -79,9 +82,6 @@ committing them to the Git repo. Unnecessary ones can be [cleared] with ## Options -> In addition to the following, `dvc exp run` accepts the options in `dvc repro` -> except for `--glob`, `--no-commit`, and `--no-run-cache`. - - `-S [:]`, `--set-param [:]` - set the value of `dvc params` for this experiment. This will update the parameters file (`params.yaml` by @@ -135,6 +135,58 @@ committing them to the Git repo. Unnecessary ones can be [cleared] with - `-m `, `--message ` - custom message to use when saving the experiment. If not provided, `dvc: commit experiment {hash}` will be used. +- `-i`, `--interactive` - ask for confirmation before reproducing each stage. + The stage is only executed if the user types "y". + +- `-s`, `--single-item` - reproduce only a single stage by turning off the + recursive search for changed dependencies. Multiple stages are executed + (non-recursively) if multiple stage names are given as `targets`. + +- `-p`, `--pipeline` - reproduce the entire pipelines that the `targets` belong + to. Use `dvc dag ` to show the parent pipeline of a target. + +- `-P`, `--all-pipelines` - reproduce all pipelines for all `dvc.yaml` files + present in the DVC project. Specifying `targets` has no effects with this + option, as all possible targets are already included. + +- `-R`, `--recursive` - looks for `dvc.yaml` files to reproduce in any + directories given as `targets`, and in their subdirectories. If there are no + directories among the targets, this option has no effect. + +- `--downstream` - only execute the stages after the given `targets` in their + corresponding pipelines, including the target stages themselves. This option + has no effect if `targets` are not provided. + +- `--force-downstream` - in cases like `... -> A (changed) -> B -> C` it will + reproduce `A` first and then `B`, even if `B` was previously executed with the + same inputs from `A` (cached). To be precise, it reproduces all descendants of + a changed stage or the stages following the changed stage, even if their + direct dependencies did not change. + + It can be useful when we have a common dependency among all stages, and want + to specify it only once (for stage `A` here). For example, if we know that all + stages (`A` and below) depend on `requirements.txt`, we can specify it in `A`, + and omit it in `B` and `C`. + + This is a way to force-execute stages without changes. This can also be useful + for pipelines containing stages that produce non-deterministic (semi-random) + outputs, where outputs can vary on each execution, meaning the cache cannot be + trusted for such stages. + +- `--pull` - attempts to download the missing dependencies of stages that need + to be run. Unless `--no-run-cache` is passed, it will also try to download the + [run cache] and the outputs of stages that are already present in it. + +- `--allow-missing` - skip stages with no other changes than missing data. + +- `-k`, `--keep-going` - Continue executing, skipping stages having dependencies + on the failed stage. The other dependencies of the targets will still be + executed. + +- `--ignore-errors` - Ignore all errors when executing the stages. Unlike + `--keep-going`, stages having dependencies on the failed stage will be + executed. + - `-h`, `--help` - prints the usage/help message, and exits. - `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if all diff --git a/content/docs/command-reference/repro.md b/content/docs/command-reference/repro.md index 9191793c56..daec7ea39c 100644 --- a/content/docs/command-reference/repro.md +++ b/content/docs/command-reference/repro.md @@ -7,11 +7,11 @@ Reproduce complete or partial pipelines by running their ```usage usage: dvc repro [-h] [-q | -v] [-f] [-i] - [-s] [-p] [-k] [-P] [-R] + [-s] [-p] [-P] [-R] [--downstream] [--force-downstream] - [--pull] [--dry] [--allow-missing] + [--pull] [--allow-missing] [--dry] [--glob] [--no-commit] [--no-run-cache] - [--ignore-errors] + [-k] [--ignore-errors] [targets [ ...]] positional arguments: