From 46e5ff32fffc095fd4bc000d56755bcf0e074250 Mon Sep 17 00:00:00 2001 From: Abdu Zoghbi Date: Wed, 18 Oct 2023 09:01:38 -0400 Subject: [PATCH] fix docs for utils.download --- docs/index.rst | 1 + docs/utils/download.rst | 50 +++++++++++++++++++++++++---------------- 2 files changed, 32 insertions(+), 19 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 7aafd6a5b..d9965de29 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -135,3 +135,4 @@ Using `pyvo` io/index auth/index utils/prototypes + utils/download diff --git a/docs/utils/download.rst b/docs/utils/download.rst index f84621856..2f36c3c5f 100644 --- a/docs/utils/download.rst +++ b/docs/utils/download.rst @@ -8,41 +8,53 @@ The download utilities provide methods for downloading data once a link to it it obtained. These can be considered an advanced version of `~pyvo.dal.Record.getdataset` that can handle data from standard on-prem servers as well as cloud data. For now only AWS is supported. -There two methods with the same call signature: `http_download` and `aws_download`. The first handles -standard links from on-prem servers, while the second downloads data from the `Amazon S3 storage`_. +There two methods with the same call signature: `~pyvo.utils.download.http_download` and `~pyvo.utils.download.aws_download`. The first handles standard links from on-prem servers, while the second downloads data from the `Amazon S3 storage`_. +.. _pyvo-download-examples: Example Usage ============== -.. code-block:: python +This is an example of downloading data when an URL or URI are available: - data_url = 'https://heasarc.gsfc.nasa.gov/FTP/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg' - image_file = http_download(url=data_url) +.. doctest-remote-data:: - s3_uri = 's3://nasa-heasarc/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg' - image2_file = aws_download(uri=data_url) + >>> from pyvo.utils import aws_download, http_download + >>> data_url = 'https://heasarc.gsfc.nasa.gov/FTP/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg' + >>> image_file = http_download(url=data_url) + >>> s3_uri = 's3://nasa-heasarc/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg' + >>> image2_file = aws_download(uri=data_url) -or -.. code-block:: python - s3_key = 'chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg's - s3_bucket = 'nasa-heasarc' - image2_file = aws_download(bucket=s3_bucket, key=s3_key) +A bucket name and a key can also be passed to `~pyvo.utils.download.aws_download` instead of an URI: +.. doctest-remote-data:: + >>> from pyvo.utils import aws_download + >>> s3_key = 'chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg' + >>> s3_bucket = 'nasa-heasarc' + >>> image2_file = aws_download(bucket=s3_bucket, key=s3_key) -If the aws data requires authentication, a credential profile (e.g. `aws_user` profile in ``~/.aws/credentials``) can be passed: -.. code-block:: python - image2_file = aws_download(bucket=s3_bucket, key=s3_key, aws_profile='aws_user') +If the aws data requires authentication, a credential profile (e.g. ``aws_user`` profile in ``~/.aws/credentials``) can be passed: +.. doctest-skip:: + >>> image2_file = aws_download(bucket=s3_bucket, key=s3_key, aws_profile='aws_user') -A session (instance of ``boto3.session.Session``) can also be passed instead (see detials in `AWS session documentation`_). -.. code-block:: python - s3_session = boto3.session.Session(aws_access_key_id, aws_secret_access_key) - image2_file = aws_download(bucket=s3_bucket, key=s3_key, session=s3_session) +A session (instance of ``boto3.session.Session``) can also be passed instead (see detials in `AWS session documentation`_): + +.. doctest-skip:: + >>> s3_session = boto3.session.Session(aws_access_key_id, aws_secret_access_key) + >>> image2_file = aws_download(bucket=s3_bucket, key=s3_key, session=s3_session) + + +.. _pyvo-download-api: + +Reference/API +============= + +.. automodapi:: pyvo.utils.download .. _Amazon S3 storage: https://aws.amazon.com/s3/