DOCS-2631: Add dataset export option CLI #3253
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viambot
added
the
safe to build
npentrel
reviewed
Aug 14, 2024
Co-authored-by: Naomi Pentrel <[email protected]>
sguequierre
commented
Aug 14, 2024
sguequierre
commented
Aug 14, 2024
docs/cli.md
Outdated
| @@ -296,6 +296,7 @@ viam data export --destination=<output path> --data-type=<output data type> [... | |||
| viam data delete [...named args] | |||
| viam data dataset add --dataset-id=<dataset-id> --file-ids=<file-id-or-ids> --location-id=<location-id> --org-id=<org-id> [...named args] | |||
| viam data dataset remove --dataset-id=<dataset-id> --file-ids=<file-id-or-ids> --location-id=<location-id> --org-id=<org-id> [...named args] | |||
| viam data dataset export --destination=<output-directory> --datasetFlagDatasetID=<dataset-id> | |||
There was a problem hiding this comment.
destination already included export in "named arguments" @tahiyasalam LMK if this needs correcting, a bit confused about flags not being renamed to simple names like destination is
There was a problem hiding this comment.
Please make sure you install the latest version of the CLI and check it to get this information:
(viam-docs) ➜ viam-docs git:(viam-agent-full) ✗ viam dataset export
NAME:
viam dataset export - download data from a dataset
USAGE:
viam dataset export --destination=<destination> --dataset-id=<dataset-id> [other options] include-jsonl parallel
OPTIONS:
--dataset-id value dataset ID of the dataset to be downloaded
--destination value output directory for downloaded data
--include-jsonl option to include JSON Lines files for local testing (default: false)
--parallel value number of download requests to make in parallel (default: 100)
Error: Required flags "destination, dataset-id" not set
Alternatively, the information is also in app.go. The lines that set the name parameters are:
- https://github.com/tahiyasalam/rdk/blob/main/cli/dataset.go#L18
- https://github.com/tahiyasalam/rdk/blob/main/cli/app.go#L75
- etc.
You can search for the variable name and see where it is getting included to find out more.
There was a problem hiding this comment.
Thank you, I wasn't aware of this and was struggling to find. Will update!
npentrel
reviewed
Aug 15, 2024
docs/cli.md
Outdated
| @@ -340,6 +341,7 @@ done | |||
| | `database hostname` | Get the MongoDB Atlas Data Federation instance hostname and connection URI. See [Configure data query](/use-cases/sensor-data-query-with-third-party-tools/#configure-data-query) | - | | |||
| | `dataset add` | Add a new image to an existing dataset by its file id, or add a group of images by specifying a filter | `filter` | | |||
| | `dataset remove` | Remove an existing image from a dataset by its file id, or remove a group of images by specifying a filter | `filter` | | |||
| | `dataset export` | Download all the data from a dataset | `datasetFlagDatasetID`: Dataset ID of the dataset to download. To retrieve these IDs, navigate to your dataset’s page in the [Viam app](https://app.viam.com), click **…** in the left-hand menu, and click **Copy dataset ID**.</li><li>`datasetFlagIncludeJSONLines`: Set to `true` to include JSON Lines files for local testing. _(optional)_</li><li>`dataFlagParallelDownloads`: Number of download requests to make in parallel. _(optional)_ Default: `100`</li></ul> | | |||
There was a problem hiding this comment.
And now that I look at this again, these are named arguments that should go in the named arguments table below. not in this table.
There was a problem hiding this comment.
yup sounds good will fix, thanks naomi
|
You can view a rendered version of the docs from this PR at https://docs-test.viam.dev/3253 |
sguequierre
changed the title
npentrel
approved these changes
Aug 16, 2024
| | `--data-type` | Data type to be downloaded: either binary or tabular |`export`| **Required** | | ||
| | `--component-name` | Filter by specified component name |`export`, `delete`| Optional | | ||
| | `--component-type` | Filter by specified component type |`export`, `delete`| Optional | | ||
| | `--component-model` | Filter by specified component model |`export`, `delete`| Optional | | ||
| | `--dataset-id` | Dataset to add or remove images from |`dataset`| **Required** | | ||
| | `--dataset-id` | Dataset to add or remove images from or download. To retrieve these IDs, navigate to your dataset’s page in the [Viam app](https://app.viam.com), click **…** in the left-hand menu, and click **Copy dataset ID**. | `dataset`, `dataset export` | **Required** | |
tahiyasalam
approved these changes
Aug 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
attempted to add explanation and usage for the flags which meant formatting a little differently in the table, lmk and I can remove