Infrastructure for creating brazil data cube environment in a local server machine
-
ds_config
: Definitions and system configs (Reference Grids, Spetral bands mapping, etc) -
ds_cubesearch
: Web interface to view datacubes, download products, etc. -
ds_executive
: Web interface for management of datacubes (definition, creating, execution) -
ds_include
: Include files needed by the system -
ds_maestro
: Docker and components that manage the cube generation system (manage queues, tasks) -
ds_opensearch
: API opensearch to the original files repository -
ds_soloist
: Docker and components that execute datacube tasks -
ds_source
: Python source codes -
ds_utils
: Python API for accessing data cube and general tools
Make sure you have the following libraries and tools installed:
Python 3
- Docker Docker version 18.06.1 or later
- Docker Compose docker-compose version 1.22.0 or later
git clone https://github.com/brazil-data-cube/bdc-scripts.git
User accounts are required to download images from providers.
In order to search and obtain images from SciHub (Copernicus) (e.g. Sentinel-2 images), users must have a registered account at: https://scihub.copernicus.eu/dhus/#/self-registration and confirm validation through email. This account may take a few days to be operational when using it in scripts.
In order to search and obtain images from Earth Explorer (USGS) (e. g. Landsat-8 images), there must be a registered account at: https://ers.cr.usgs.gov/register/ and confirm validation through email.
In order to perform LaSRC atmospheric correction on Landsat-8 images, several auxiliary files are required. LADS are auxiliare files produced using MODIS data. in order to obtain LADS users must have an account at Earth Data: https://urs.earthdata.nasa.gov/home, in order to obtain a token (after the login) to use in the scripts.
In order to upload the files to AWS S3, AWS accessKeys are also required.
insert your SciHub user and password in rc_maestro/secrets_s2.csv insert your SciHub user and password in rc_maestro/secrets_S2.JSON insert your USGS user and password in rc_maestro/secrets_USGS.csv insert your AWS user and password in rc_maestro/accessKeys.csv
modify bdc_compose.yml (set ports, passwords and mount volumes)
** If landsat atmospheric correction (LaSRC) is going to be performed, user must run updateLADS.py to obtain auxiliary data for each day of the year.
docker-compose -f bdc_compose.yml up
** stop docker-compes: docker-compose -f bdc_compose.yml down
open PHPmy admin from browser setting its path/port and import tables to your db (sql files)
[Download
] -> [Atm correction
] -> [publish
] -> [upload
]
[warp
] -> [merge
] -> [blend
]
curl :/radcor?w=-45.90&s=-12.74&n=-12.6&e=-45.80&satsen=S2&start=2018-09-01&end=2018-09-31
Consult available images: curl :/radcor?w=-45.90&s=-12.74&n=-12.6&e=-45.80&satsen=LC8&start=2018-09-01&end=2018-09-31
Download available images: curl :/radcor?w=-45.90&s=-12.74&n=-12.6&e=-45.80&satsen=LC8&start=2018-09-01&end=2018-09-31&action=process
curl :/create?datacube=L30m&satsen=LC8&start=2018-09-01&end=2019-08-31&wrs=aea_250k&resx=30&resy=30&tschema=M&bands=coastal,blue,green,red,nir,swir1,swir2,ndvi,evi,quality&quicklook=swir2,nir,red
curl :/create?datacube=S1016d&satsen=S2SR&start=2018-09-01&end=2019-08-31&wrs=aea_250k&resx=10&resy=10&tschema=A&step=16&bands=coastal,blue,green,red,redge1,redge2,redge3,nir,bnir,swir1,swir2,ndvi,evi,quality&quicklook=swir2,nir,red
curl :/process?datacube=L30m&pr=089098&start=start=2018-09-01&end=2019-08-31
curl :/deldice?datacube=L30m&tileid=089098&startdate=2019-04-01