Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using pystac and pystac-client for reading a STAC catalog and implementing StacDataStore.get_data_ids() and StacDataStore.has_data() #9

Closed
wants to merge 11 commits into from
29 changes: 29 additions & 0 deletions .github/workflows/unitest-workflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Build

on:
push:
release:
types: [published]

jobs:
unittest:
runs-on: ubuntu-latest
env:
NUMBA_DISABLE_JIT: 1
steps:
- name: checkout xcube-stac
uses: actions/checkout@v3

- name: Set up MicroMamba
uses: mamba-org/provision-with-micromamba@main
with:
cache-env: true
extra-specs: |
python=3.10

- name: Run unit tests
shell: bash -l {0}
run: |
cd /home/runner/work/xcube-stac/xcube-stac
ls
pytest
114 changes: 114 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Credentials
*-credentials.json

# Test output data
test-outputs/

# IntelliJ / PyCharm
.idea

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.vscode/settings.json
3 changes: 3 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## Changes in 0.1.0 (in development)

Initial version of STAC Data Store.
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 Brockmann Consult GmbH

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
15 changes: 15 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: xcube-stac
channels:
- conda-forge
- defaults
dependencies:
# Required
- pystac
- pystac-client
- xarray
- xcube
# for testing
- black
- flake8
- pytest
- pytest-recording
48 changes: 48 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
[build-system]
requires = ["setuptools >= 61.2.0"]
build-backend = "setuptools.build_meta"

[project]
name = "xcube_stac"
dynamic = ["version"]
authors = [
{name = "Konstantin Ntokas", email = "[email protected]"}
]
description = """\
xcube data store for accessing stac catalogs\
"""
keywords = [
"analysis ready data", "data science", "datacube", "xarray", "zarr", "xcube", "stac"
]
readme = {file = "README.md", content-type = "text/markdown"}
license = {text = "MIT"}
requires-python = ">=3.10"
dependencies = [
"pystac",
"pystac-client",
"xarray",
"xcube"

]

[tool.setuptools.dynamic]
version = {attr = "xcube_stac.__version__"}

[tool.setuptools.packages.find]
exclude = [
"test*",
"doc*"
]

[project.optional-dependencies]
dev = [
"pytest",
"pytest-cov",
"black",
"flake8"
]

[project.urls]
Repository = 'https://github.com/xcube-dev/xcube-stac'
Issues = 'https://github.com/xcube-dev/xcube-stac/issues'
Changelog = 'https://github.com/xcube-dev/xcube-stac/blob/main/CHANGES.md'
Empty file added test/__init__.py
Empty file.
150 changes: 150 additions & 0 deletions test/cassettes/test_stac/StacTest.test_open_data.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
interactions:
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate, br
Connection:
- keep-alive
User-Agent:
- python-requests/2.31.0
method: GET
uri: https://raw.githubusercontent.com/stac-extensions/label/main/examples/multidataset/catalog.json
response:
body:
string: "{\n \"stac_version\": \"1.0.0-rc.1\",\n \"type\": \"Catalog\",\n
\ \"id\": \"label_extension_demo\",\n \"title\": \"label extension demo\",\n
\ \"description\": \"Sample ML training data labels in the STAC format\",\n
\ \"links\": [\n {\n \"rel\": \"root\",\n \"href\": \"./catalog.json\"\n
\ },\n {\n \"rel\": \"child\",\n \"href\": \"zanzibar/collection.json\"\n
\ },\n {\n \"rel\": \"child\",\n \"href\": \"spacenet-buildings/collection.json\"\n
\ }\n ]\n}"
headers:
Accept-Ranges:
- bytes
Access-Control-Allow-Origin:
- '*'
Cache-Control:
- max-age=300
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Length:
- '236'
Content-Security-Policy:
- default-src 'none'; style-src 'unsafe-inline'; sandbox
Content-Type:
- text/plain; charset=utf-8
Cross-Origin-Resource-Policy:
- cross-origin
Date:
- Mon, 29 Apr 2024 09:40:39 GMT
ETag:
- W/"acb7a8d6636e24e32f4018c14f1c4ff418a82567b2746560f9eae6ad97a48a54"
Expires:
- Mon, 29 Apr 2024 09:45:39 GMT
Source-Age:
- '37'
Strict-Transport-Security:
- max-age=31536000
Vary:
- Authorization,Accept-Encoding,Origin
Via:
- 1.1 varnish
X-Cache:
- HIT
X-Cache-Hits:
- '3'
X-Content-Type-Options:
- nosniff
X-Fastly-Request-ID:
- 1344da34a305e1ad724b5ff5775b6496d4340ff3
X-Frame-Options:
- deny
X-GitHub-Request-Id:
- AAE2:26494:2404BC4:260086D:662F654D
X-Served-By:
- cache-cph2320046-CPH
X-Timer:
- S1714383639.266545,VS0,VE0
X-XSS-Protection:
- 1; mode=block
status:
code: 200
message: OK
- request:
body: null
headers:
Connection:
- close
Host:
- raw.githubusercontent.com
User-Agent:
- Python-urllib/3.12
method: GET
uri: https://raw.githubusercontent.com/stac-extensions/label/main/examples/multidataset/catalog.json
response:
body:
string: "{\n \"stac_version\": \"1.0.0-rc.1\",\n \"type\": \"Catalog\",\n
\ \"id\": \"label_extension_demo\",\n \"title\": \"label extension demo\",\n
\ \"description\": \"Sample ML training data labels in the STAC format\",\n
\ \"links\": [\n {\n \"rel\": \"root\",\n \"href\": \"./catalog.json\"\n
\ },\n {\n \"rel\": \"child\",\n \"href\": \"zanzibar/collection.json\"\n
\ },\n {\n \"rel\": \"child\",\n \"href\": \"spacenet-buildings/collection.json\"\n
\ }\n ]\n}"
headers:
Accept-Ranges:
- bytes
Access-Control-Allow-Origin:
- '*'
Cache-Control:
- max-age=300
Connection:
- close
Content-Length:
- '436'
Content-Security-Policy:
- default-src 'none'; style-src 'unsafe-inline'; sandbox
Content-Type:
- text/plain; charset=utf-8
Cross-Origin-Resource-Policy:
- cross-origin
Date:
- Mon, 29 Apr 2024 09:40:39 GMT
ETag:
- '"e74ebcbc46d43c5b693ecb995381fbeba03583627e6d65b21ed7678a10d94729"'
Expires:
- Mon, 29 Apr 2024 09:45:39 GMT
Source-Age:
- '37'
Strict-Transport-Security:
- max-age=31536000
Vary:
- Authorization,Accept-Encoding,Origin
Via:
- 1.1 varnish
X-Cache:
- HIT
X-Cache-Hits:
- '1'
X-Content-Type-Options:
- nosniff
X-Fastly-Request-ID:
- 7e45b04e3ce790c6f2b04d14e258ae95f8080c52
X-Frame-Options:
- deny
X-GitHub-Request-Id:
- AAE2:26494:2404BDD:2600886:662F654D
X-Served-By:
- cache-cph2320033-CPH
X-Timer:
- S1714383639.317881,VS0,VE1
X-XSS-Protection:
- 1; mode=block
status:
code: 200
message: OK
version: 1
Loading
Loading