Skip to content

Commit

Permalink
fix docs for utils.download
Browse files Browse the repository at this point in the history
  • Loading branch information
zoghbi-a committed Oct 18, 2023
1 parent 2dcbe7b commit 46e5ff3
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 19 deletions.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,4 @@ Using `pyvo`
io/index
auth/index
utils/prototypes
utils/download
50 changes: 31 additions & 19 deletions docs/utils/download.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,41 +8,53 @@ The download utilities provide methods for downloading data once a link to it
it obtained. These can be considered an advanced version of `~pyvo.dal.Record.getdataset` that can handle
data from standard on-prem servers as well as cloud data. For now only AWS is supported.

There two methods with the same call signature: `http_download` and `aws_download`. The first handles
standard links from on-prem servers, while the second downloads data from the `Amazon S3 storage`_.
There two methods with the same call signature: `~pyvo.utils.download.http_download` and `~pyvo.utils.download.aws_download`. The first handles standard links from on-prem servers, while the second downloads data from the `Amazon S3 storage`_.


.. _pyvo-download-examples:

Example Usage
==============
.. code-block:: python
This is an example of downloading data when an URL or URI are available:

data_url = 'https://heasarc.gsfc.nasa.gov/FTP/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg'
image_file = http_download(url=data_url)
.. doctest-remote-data::

s3_uri = 's3://nasa-heasarc/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg'
image2_file = aws_download(uri=data_url)
>>> from pyvo.utils import aws_download, http_download
>>> data_url = 'https://heasarc.gsfc.nasa.gov/FTP/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg'
>>> image_file = http_download(url=data_url)

>>> s3_uri = 's3://nasa-heasarc/chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg'
>>> image2_file = aws_download(uri=data_url)

or
.. code-block:: python

s3_key = 'chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg's
s3_bucket = 'nasa-heasarc'
image2_file = aws_download(bucket=s3_bucket, key=s3_key)
A bucket name and a key can also be passed to `~pyvo.utils.download.aws_download` instead of an URI:

.. doctest-remote-data::
>>> from pyvo.utils import aws_download
>>> s3_key = 'chandra/data/byobsid/2/3052/primary/acisf03052N004_cntr_img2.jpg'
>>> s3_bucket = 'nasa-heasarc'
>>> image2_file = aws_download(bucket=s3_bucket, key=s3_key)

If the aws data requires authentication, a credential profile (e.g. `aws_user` profile in ``~/.aws/credentials``) can be passed:
.. code-block:: python

image2_file = aws_download(bucket=s3_bucket, key=s3_key, aws_profile='aws_user')
If the aws data requires authentication, a credential profile (e.g. ``aws_user`` profile in ``~/.aws/credentials``) can be passed:

.. doctest-skip::
>>> image2_file = aws_download(bucket=s3_bucket, key=s3_key, aws_profile='aws_user')

A session (instance of ``boto3.session.Session``) can also be passed instead (see detials in `AWS session documentation`_).
.. code-block:: python

s3_session = boto3.session.Session(aws_access_key_id, aws_secret_access_key)
image2_file = aws_download(bucket=s3_bucket, key=s3_key, session=s3_session)
A session (instance of ``boto3.session.Session``) can also be passed instead (see detials in `AWS session documentation`_):

.. doctest-skip::
>>> s3_session = boto3.session.Session(aws_access_key_id, aws_secret_access_key)
>>> image2_file = aws_download(bucket=s3_bucket, key=s3_key, session=s3_session)


.. _pyvo-download-api:

Reference/API
=============

.. automodapi:: pyvo.utils.download


.. _Amazon S3 storage: https://aws.amazon.com/s3/
Expand Down

0 comments on commit 46e5ff3

Please sign in to comment.