-
Notifications
You must be signed in to change notification settings - Fork 392
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
12 changed files
with
385 additions
and
34 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
# dvc.api.artifacts_show() | ||
|
||
Get the path and Git revision for an <abbr>artifact</abbr> tracked in a | ||
<abbr>DVC repository</abbr>. | ||
|
||
```py | ||
def artifacts_show( | ||
name: str, | ||
version: Optional[str] = None, | ||
stage: Optional[str] = None, | ||
repo: Optional[str] = None, | ||
) -> Dict[str, str]: | ||
``` | ||
|
||
## Usage: | ||
|
||
```py: | ||
import dvc.api | ||
artifact = dvc.api.artifacts_show( | ||
'text-classification', | ||
repo='https://github.com/iterative/example-get-started.git', | ||
) | ||
``` | ||
|
||
## Description | ||
|
||
Returns a path and Git revision for a named artifact which can then be used in | ||
other Python API calls. | ||
|
||
The returned dictionary will be of the form: | ||
|
||
```py | ||
{ | ||
'path': 'model.pkl', | ||
'rev': 'c7c6ae0', | ||
} | ||
``` | ||
|
||
where `path` contains the relative path to the artifact in the DVC repository, | ||
and `rev` contains the Git revision for the specified artifact version or stage. | ||
|
||
When neither `version` nor `stage` are provided, the Git revision for the latest | ||
version of the model will be returned. | ||
|
||
## Parameters | ||
|
||
- `name` (required) - name of the artifact. By default DVC will search for | ||
artifacts declared in a `dvc.yaml` file located at the root of the DVC | ||
repository. Artifacts declared in other `dvc.yaml` files should be addressed | ||
in the form `path/to/dvc.yaml:artifact_name` or `path/to:artifact_name` (where | ||
`dvc.yaml` is omitted). | ||
|
||
- `version` - version of the artifact (mutually exclusive with `stage`). | ||
|
||
- `stage` - stage of the artifact (mutually exclusive with `version`). | ||
|
||
- `repo` - the location of the DVC project. It can be a URL or a file system | ||
path. Both HTTP and SSH protocols are supported for online Git repos (e.g. | ||
`[user@]server:project.git`). _Default_: The current project (found by walking | ||
up from the current working directory tree). | ||
|
||
## Example: Read the contents of an artifact | ||
|
||
```py | ||
import pickle | ||
import dvc.api | ||
|
||
artifact = dvc.api.artifacts_show( | ||
'text-classification', | ||
version='v1.0.0', | ||
repo='https://github.com/iterative/example-get-started.git', | ||
) | ||
data = dvc.api.read( | ||
artifact['path'], | ||
rev=artifact['rev'], | ||
repo='https://github.com/iterative/example-get-started.git', | ||
mode='rb', | ||
) | ||
model = pickle.loads(data) | ||
``` | ||
|
||
This example uses the returned path and Git revision in conjunction with | ||
`dvc.api.read()` to read the file content for the artifact. | ||
|
||
## Example: Download an artifact | ||
|
||
```py | ||
import os | ||
import dvc.api | ||
|
||
artifact = dvc.api.artifacts_show( | ||
'text-classification', | ||
stage='prod', | ||
repo='https://github.com/iterative/example-get-started.git', | ||
) | ||
fs = dvc.api.DVCFileSystem( | ||
'https://github.com/iterative/example-get-started.git', | ||
rev=artifact['rev'], | ||
) | ||
fs.get_file(artifact['path'], os.path.basename(artifact['path'])) | ||
``` | ||
|
||
This example uses the returned path and Git revision in conjunction with | ||
`dvc.api.DVCFileSystem` to download the artifact to the current working | ||
directory. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
## artifacts get | ||
|
||
Download an <abbr>artifact</abbr> tracked in a DVC project into the current | ||
working directory. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: dvc artifacts get [-h] [-q | -v] | ||
[--rev [<version>]] [--stage [<stage>]] | ||
[-o [<path>]] [-j <number>] [-f] | ||
[--config CONFIG] | ||
[--remote REMOTE] [--remote-config [REMOTE_CONFIG ...]] | ||
url name | ||
positional arguments: | ||
url Location of DVC repository to download from | ||
name Name of artifact in the repository | ||
``` | ||
|
||
## Description | ||
|
||
Provides a way to download artifacts tracked in a DVC project. Unlike `dvc get`, | ||
`dvc artifacts get` supports downloading an artifact by name, rather than by | ||
path. Likewise, `dvc artifacts get` supports downloading a registered artifact | ||
version or stage, instead of requiring a specified Git revision. | ||
|
||
`dvc artifacts get` also supports downloading artifacts both from the | ||
<abbr>model registry</abbr> and from DVC remotes. | ||
|
||
<admon type="tip"> | ||
|
||
Downloading an artifact from the <abbr>model registry</abbr> only requires a | ||
valid Studio | ||
[access token](/doc/studio/user-guide/account-management#studio-access-token). | ||
It does not require the client to have DVC remote credentials. | ||
|
||
</admon> | ||
|
||
The `url` argument specifies the address of the DVC or Git repository containing | ||
the artifact. Both HTTP and SSH protocols are supported (e.g. | ||
`[user@]server:project.git`). `url` can also be a local file system path | ||
(including the current project e.g. `.`). | ||
|
||
The `name` argument specifies the name of the artifact to download. By default | ||
DVC will search for artifacts declared in a `dvc.yaml` file located at the root | ||
of the DVC repository. Artifacts declared in other `dvc.yaml` files should be | ||
addressed in the form `path/to/dvc.yaml:artifact_name` or | ||
`path/to:artifact_name` (where `dvc.yaml` is omitted). | ||
|
||
<admon type="info"> | ||
|
||
`dvc artifacts get` will first try to download artifacts via the <abbr>model | ||
registry</abbr>. If you do not have a valid Studio token, or the artifact is not | ||
tracked in the model registry, DVC will fall back to downloading the artifact | ||
from the project's default DVC remote. | ||
|
||
</admon> | ||
|
||
## Options | ||
|
||
- `--rev <version>` - Version of the artifact to download. The latest version of | ||
the artifact is used by default when neither `rev` nor `stage` are specified. | ||
|
||
- `--stage <stage>` - Stage of the artifact to download. The latest version of | ||
the artifact is used by default when neither `rev` nor `stage` are specified. | ||
|
||
- `-o <path>`, `--out <path>` - specify a `path` to the desired location in the | ||
workspace to place the downloaded file or directory (instead of using the | ||
current working directory). Directories specified in the path will be created | ||
by this command. | ||
|
||
- `-j <number>`, `--jobs <number>` - parallelism level for DVC to download data | ||
from the remote. The default value is `4 * cpu_count()`. Using more jobs may | ||
speed up the operation. Note that the default value can be set in the source | ||
repo using the `jobs` config option of `dvc remote modify`. | ||
|
||
- `-f`, `--force` - when using `--out` to specify a local target file or | ||
directory, the operation will fail if those paths already exist. this flag | ||
will force the operation causing local files/dirs to be overwritten by the | ||
command. | ||
|
||
- `--config <path>` - path to a [config file](/doc/command-reference/config) | ||
that will be merged with the config in the target repository. | ||
|
||
- `--remote <name>` - name of the `dvc remote` to set as a default in the target | ||
repository. Only applicable when downloading artifacts from a DVC remote. | ||
|
||
- `--remote-config [<name>=<value> ...]` - `dvc remote` config options to merge | ||
with a remote's config (default or one specified by `--remote`) in the target | ||
repository. Only applicable when downloading artifacts from a DVC remote. | ||
|
||
- `-h`, `--help` - prints the usage/help message, and exit. | ||
|
||
- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no | ||
problems arise, otherwise 1. | ||
|
||
- `-v`, `--verbose` - displays detailed tracing information. | ||
|
||
## Example: Download an artifact from a DVC remote | ||
|
||
```cli | ||
$ dvc artifacts get https://github.com/iterative/example-get-started.git text-classification --rev=v1.0.0 | ||
Downloaded 1 file(s) to 'model.pkl' | ||
``` | ||
|
||
In this example, we download version `v1.0.0` of the artifact. Since we have no | ||
Studio credentials set in our environment, `dvc artifacts get` will download the | ||
artifact from the default DVC remote defined in the repository. | ||
|
||
## Example: Download an artifact using a Studio token | ||
|
||
```cli | ||
$ DVC_STUDIO_TOKEN=mytoken dvc artifacts get https://github.com/iterative/example-get-started.git text-classification --stage=prod | ||
Downloaded 1 file(s) to 'model.pkl' | ||
``` | ||
|
||
In this example, we download stage `prod` of the artifact. Since we have set our | ||
Studio access token in the `DVC_STUDIO_TOKEN` environment variable, | ||
`dvc artifacts get` will download the artifact via the <abbr>model | ||
registry</abbr> rather than from a DVC remote. | ||
|
||
## Example: Download an artifact defined in a specific `dvc.yaml` file | ||
|
||
```cli | ||
$ dvc artifacts get https://github.com/iterative/lstm_seq2seq.git results/dvc.yaml:best | ||
Downloaded 1 file(s) to 'epoch=0-step=16.ckpt' | ||
``` | ||
|
||
In this example, we download the latest version of the `best` artifact. In this | ||
case, the artifact is defined in `results/dvc.yaml` so we must include the path | ||
to the `dvc.yaml` file when addressing the artifact. Since we do not specify | ||
`--rev` or `--stage`, `dvc artifacts get` will download the latest version of | ||
the artifact by default. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# artifacts | ||
|
||
Commands for working with DVC <abbr>artifacts</abbr> and the <abbr>model | ||
registry</abbr>. | ||
|
||
## Synopsis | ||
|
||
```usage | ||
usage: dvc artifacts [-h] [-q | -v] {get} ... | ||
positional arguments: | ||
COMMAND | ||
get Download an artifact from a DVC project. | ||
``` | ||
|
||
## Description | ||
|
||
`dvc artifacts` subcommands provide a command line client for working with | ||
<abbr>model registry</abbr> artifacts. | ||
|
||
## Options | ||
|
||
- `-h`, `--help` - prints the usage/help message, and exit. | ||
|
||
- `-q`, `--quiet` - do not write anything to standard output. | ||
|
||
- `-v`, `--verbose` - displays detailed tracing information. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.