Skip to content

Commit

Permalink
chore: readme description
Browse files Browse the repository at this point in the history
  • Loading branch information
renaudjester committed Jun 28, 2024
1 parent 723967b commit 2ab531b
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 2 deletions.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,17 +92,22 @@ The toolbox makes many requests to STAC to be able to parse the full marine data

Note, that this concerns only the catalog parsing step so the describe command and the start of the get and subset command. It does not apply when downloading files or listing files from the get command or when requesting the data chunks for the subset command.

For the `get` command, you can use the `COPERNICUSMARINE_GET_CONCURRENT_DOWNLOADS` to set the number of threads open to download in parallel. There are no default value. By default the toolbox uses the python `multiprocessing.pool.ThreadPool`. You can set the environment variable to 0 if you don't want to use the `multiprocessing` library at all, the download will be used only through `boto3`.

## Command Line Interface (CLI)

### The `--help` option

To discover commands and their available options, consider appending `--help` on any command line.

Example:

```bash
copernicusmarine --help
```

Returns:

```bash
Usage: copernicusmarine [OPTIONS] COMMAND [ARGS]...

Expand All @@ -118,20 +123,25 @@ Commands:
```

### Command `describe`

Retrieve metadata information about all products/datasets and display as JSON output:

```bash
copernicusmarine describe --include-datasets
```

The JSON output can also be saved as follows:

```bash
copernicusmarine describe --include-datasets > all_datasets_copernicusmarine.json
```

### Command `login`

Create a single configuration file `.copernicusmarine-credentials` allowing to access all Copernicus Marine Data Store data services. By default, the file is saved in user's home directory.

Example:

```bash
> copernicusmarine login
username : johndoe
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,10 @@ def download_files(
if not parent_dir.is_dir():
pathlib.Path.mkdir(parent_dir, parents=True)

# TODO: It would be proably better to use an async approach
# TODO: v2 It would be proably better to use an async approach
# TODO: v2 probably better to use an argument for the number
# of threads instead of using the environment variable
if NUMBER_THREADS is None or NUMBER_THREADS:
logger.info(f"Downloading files on {NUMBER_THREADS} threads...")
pool = ThreadPool(processes=NUMBER_THREADS)
download_summary_list: Iterator[List[Path]] = pool.imap(
_download_files,
Expand Down

0 comments on commit 2ab531b

Please sign in to comment.