Skip to content

Commit

Permalink
feat: nisar sim v2 (#8)
Browse files Browse the repository at this point in the history
* feat: simulated NISAR v2 - base
* chore: updated examples
* fix: small change to hrefs and access
* test: updated tests, mocked data file
* chore: updated descriptions
* docs: updated documentation
  • Loading branch information
jjfrench authored Mar 6, 2024
1 parent 97ce6ba commit 2b6dd48
Show file tree
Hide file tree
Showing 33 changed files with 717 additions and 5,168 deletions.
16 changes: 14 additions & 2 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,19 @@
"DOCKER_WORKDIR": "/workspace"
}
},

// Attempting to add mounts to AWS credentials for development environment
"mounts": [
{
"source": "${localEnv:HOME}/.aws",
"target": "/root/.aws",
"type": "bind"
},
{
"source": "${localEnv:HOME}/.ssh",
"target": "/root/.ssh",
"type": "bind"
}
],
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
Expand All @@ -30,7 +42,7 @@
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance"
"ms-toolsai.jupyter"
]

// Use 'forwardPorts' to make a list of ports inside the container available locally.
Expand Down
5 changes: 5 additions & 0 deletions .github/.markdownlint-cli2.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"config": {
"MD041": false
}
}
16 changes: 0 additions & 16 deletions .github/PULL_REQUEST_TEMPLATE.MD

This file was deleted.

7 changes: 7 additions & 0 deletions .markdownlint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Default state for all rules
default: true

# MD013/line-length - Line length
MD013:
line_length: 80
code_blocks: false
1 change: 0 additions & 1 deletion .markdownlintignore

This file was deleted.

25 changes: 11 additions & 14 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,29 @@
# Please run `pre-commit run --all-files` when adding or changing entries.

repos:
- repo: https://github.com/psf/black
rev: 23.7.0
- repo: https://github.com/DavidAnson/markdownlint-cli2
rev: v0.9.2
hooks:
- id: black
- id: markdownlint-cli2
language_version: "18.17.1"
- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
hooks:
- id: codespell
args: [--ignore-words=.codespellignore]
types_or: [jupyter, markdown, python, shell]
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
- repo: https://github.com/psf/black
rev: 23.9.1
hooks:
- id: flake8
- id: black
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.4.1
rev: v1.5.1
hooks:
- id: mypy
additional_dependencies:
- click != 8.1.0
- stactools
- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.35.0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.288
hooks:
- id: markdownlint
- id: ruff
37 changes: 11 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,20 @@
# stactools-nisar-sim

[![PyPI](https://img.shields.io/pypi/v/stactools-nisar-sim)](https://pypi.org/project/stactools-nisar-sim/)

- Name: nisar-sim
- Package: `stactools.nisar_sim`
- [stactools-nisar-sim on PyPI](https://pypi.org/project/stactools-nisar-sim/)
- Owner: @jjfrench @wildintellect
- [Dataset homepage](https://uavsar.jpl.nasa.gov/science/documents/nisar-sample-products.html)
- STAC extensions used:
- [proj](https://github.com/stac-extensions/projection/)
- [sat](https://github.com/stac-extensions/sat)
- [Browse the example in human-readable form](https://radiantearth.github.io/stac-browser/#/external/raw.githubusercontent.com/stactools-packages/nisar-sim/main/examples/collection.json)
- [alternative-assets](https://stac-extensions.github.io/alternate-assets/v1.1.0/schema.json)

stactools package for use with the simulated NISAR products.

## STAC Examples

- [Collection](examples/collection.json)
- [Item](examples/winnip_31604_12061_004_120717_L090_CX_07/winnip_31604_12061_004_120717_L090_CX_07.json)
- [Item](/examples/NISAR_L0_PR_RRSD_001_005_A_128S_20081012T060910_20081012T060926_P01101_F_J_001/NISAR_L0_PR_RRSD_001_005_A_128S_20081012T060910_20081012T060926_P01101_F_J_001.json)

## Installation

Expand All @@ -35,27 +32,10 @@ stac nisarsim create-collection <destination/>
stac nisarsim create-collection example-collection.json
```

**Dither**: Indicates whether the data has been dithered or not:

- "X": not dithered,
- "G": dithered with gaps,
- "D": dithered with no gaps.

**Nmode**: NISAR mode, a 3-digit mode number associated a specific center
frequency, bandwidth, and polarization.

- "129"
- "138"
- "143"

Read more about the _Dither_ and _Nmode_ namings [here](https://uavsar.jpl.nasa.gov/science/documents/nisar-sample-products.html#Product%20Description:~:text=about%20dithered%20products.-,Naming%20Convention,-The%20simulated%20NISAR).

```shell
stac nisarsim create-item <source/> <destination/> --dither <dither> --nmode <nmode>
stac nisarsim create-item <source/> <destination/>

stac nisarsim create-item
https://downloaduav.jpl.nasa.gov/Release2v/winnip_31604_12061_004_120717_L090_CX_07/
example.json --dither X --nmode 129
stac nisarsim create-item "https://nisar.asf.earthdatacloud.nasa.gov/NISAR-SAMPLE-DATA/L0B/ALOS1_Rosamond_20081012/NISAR_L0_PR_RRSD_001_005_A_128S_20081012T060910_20081012T060926_P01101_F_J_001.h5" examples/NISAR_L0_PR_RRSD_001_005_A_128S_20081012T060910_20081012T060926_P01101_F_J_001.json
```

Use `stac nisarsim --help` to see all subcommands and options.
Expand All @@ -66,8 +46,7 @@ We use [pre-commit](https://pre-commit.com/) to check any changes.
To set up your development environment:

```shell
pip install -e .
pip install -r requirements-dev.txt
pip install -e '.[dev]'
pre-commit install
```

Expand All @@ -82,3 +61,9 @@ To run the tests:
```shell
./scripts/test
```

If you've updated the STAC metadata output, update the examples:

```shell
scripts/update-examples
```
29 changes: 12 additions & 17 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,51 +8,46 @@ RUN conda update conda && \

FROM base as dependencies

ENV PATH="/opt/venv/bin:$PATH"
ENV PATH="/opt/conda/bin:$PATH"
WORKDIR /opt/stactools-nisar-sim
COPY --from=base /opt/conda /opt/conda
COPY pyproject.toml setup.cfg ./
COPY pyproject.toml ./
COPY src/stactools/nisar_sim/__init__.py src/stactools/nisar_sim/
RUN apt-get -y -q update \
&& apt-get -y -q install build-essential \
&& rm -rf /var/lib/apt/lists/
RUN python -m venv /opt/venv
RUN pip install . h5py --no-binary rasterio \
&& ls /opt/venv \
&& rm -r /opt/venv/lib/python3.11/site-packages/stactools/nisar_sim
RUN pip install .
RUN rm -r /opt/conda/lib/python3.11/site-packages/stactools/nisar_sim


FROM dependencies as builder

ENV PATH="/opt/venv/bin:$PATH"
ENV PATH="/opt/conda/bin:$PATH"
WORKDIR /opt/stactools-nisar-sim
COPY --from=base /opt/conda /opt/conda
COPY --from=dependencies /opt/venv /opt/venv
COPY pyproject.toml setup.cfg ./
COPY pyproject.toml ./
COPY src ./src
RUN pip install -U pip \
&& pip install . h5py --no-binary rasterio
&& pip install .
WORKDIR /
RUN rm -rf /opt/stactools-nisar-sim
CMD [ "stac", "nisarsim" ]


FROM dependencies as dev-dependencies

ENV PATH="/opt/venv/bin:$PATH"
ENV PATH="/opt/conda/bin:$PATH"
WORKDIR /opt/stactools-nisar-sim
COPY --from=dependencies /opt/venv /opt/venv
COPY requirements-dev.txt .
RUN pip install -r requirements-dev.txt \
COPY --from=dependencies /opt/conda /opt/conda
COPY pyproject.toml .
RUN pip install -e '.[dev]' \
&& git init


FROM dev-dependencies as dev-builder

ENV PATH="/opt/venv/bin:$PATH"
ENV PATH="/opt/conda/bin:$PATH"
WORKDIR /opt/stactools-nisar-sim
COPY --from=base /opt/conda /opt/conda
COPY --from=dependencies /opt/venv /opt/venv
COPY . .
RUN pip install -e .
CMD [ "stac", "nisarsim" ]
Loading

0 comments on commit 2b6dd48

Please sign in to comment.