Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for local testing of tile cache server #194

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 33 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,19 @@ Raster and vector tile API for datasets in the [GFW Data API](https://github.com

## Developing
### Option 1: Developing against the Data API postgres database in one of the cloud environments (dev, staging or production):
* Make sure you have ssh access to the bastion host of the aws account (contact a Data API Engineering team member to get help with this).
* Make sure you have ssh access to the bastion host of the aws account (contact a
Data API Engineering team member to get help with this). Also, your host IP address
(obtained by visiting `whatsmyip.com`) should be whitelisted for port 22 access in
the gfw-production core-default security group (at
`https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#SecurityGroups:`)

* Open ssh tunnel connection to the database you'd like to connect to. For example, for the `staging` environment:
* Open an ssh tunnel connection to the database you'd like to connect to. For example, for the `staging` environment:

```ssh -i ~/.ssh/id_rsa -N -L 5432:application-autoscaling-698c9c01-db99-4430-a97a-6baaae853dc6.cljrqsduwhdo.us-east-1.rds.amazonaws.com:5432 ec2-user@gfw-staging```
```ssh -i ~/.ssh/id_rsa -N -L 5432:application-autoscaling-698c9c01-db99-4430-a97a-6baaae853dc6.cljrqsduwhdo.us-east-1.rds.amazonaws.com:5432 ec2-user@{bastion_host}```

* Set the environment variables for the **read only** credentials of the above database. The environment variables are `GFW_DB_NAME`, `GFW_DB_USER_RO` and `GFW_DB_PASSWORD_RO`. These are also listed in the `docker-compose.dev.yml` file.
{bastion_host} might be something like: `ec2-18-215-196-0.compute-1.amazonaws.com`

* Set the environment variables for the **read only** credentials of the above database. The environment variables are `GFW_DB_NAME`, `GFW_DB_USER_RO`, `GFW_DB_PASSWORD_RO`, and `PLANET_API_KEY`. These are also listed in the `docker-compose.dev.yml` file. You get the values of secrets from the secrets manager of the appropriate AWS environment (production, staging, etc.)

* In `docker-compose.dev.yml`, set `DATA_LAKE_BUCKET` to the desired environment's bucket name. By default, the `staging` environment bucket (`gfw-data-lake-staging`) will be used.

Expand All @@ -18,9 +24,31 @@ Raster and vector tile API for datasets in the [GFW Data API](https://github.com
* Run the start up script from the root directory:
```./scripts/develop```

You access the tile cache server at `localhost:8088` on your host.


NOTE: if you are developing on Linux (as opposed to MacOS), there can be a problem
accessing the host network, so `scripts/develop` will likely hang without being able
to access the database. If so, you can interrupt `scripts/develop` after it has built
the docker and gotten hung accessing the database. You can then start the docker by
hand via:

```docker run -it -p 127.0.0.1:8088:80 --entrypoint /bin/bash gfw-tile-cache_app:latest```

In that case, you will need to copy your `.ssh/id_rsa` and `.aws/credentials` files
into the docker and set all the needed environmental variables in the shell. You then
run the ssh tunnel from inside the docker via:

```ssh -i ~/.ssh/id_rsa -N -L 127.0.0.1:5432:gfw-aurora.cluster-ro-ch3jv7fz9pj1.us-east-1.rds.amazonaws.com:5432 "[email protected]" &```

You can then start up the tile cache server in the docker via:

```/start-reload.sh &```


### Option 2: Developing against a local instance of Data API database

* Start dev instance of Data API locally using the instructions [here](https://github.com/wri/gfw-data-api?tab=readme-ov-file#run-locally-with-docker)

* Run the start up script from the root directory with the option to point to the local Data API:
```./scripts/develop --local_data_api```
```./scripts/develop --local_data_api```
4 changes: 2 additions & 2 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ services:
- LOG_LEVEL=debug
- RASTER_TILER_LAMBDA_NAME=test
- ENV=dev
- PLANET_API_KEY
- PLANET_API_KEY=${PLANET_API_KEY}
- AWS_REQUEST_PAYER=requester
- 'DIST_ALERTS_FOREST_FILTERS={"tree_cover_loss": {"dataset": "umd_tree_cover_loss", "version": "v1.10.1"}, "tree_cover_height": {"dataset": "umd_tree_cover_height_2020", "version": "v2022"}, "tree_cover_density": {"dataset": "umd_tree_cover_density_2010", "version": "v1.6"}}'
ports:
- 8088:80
- 127.0.0.1:8088:80
entrypoint: wait_for_postgres.sh /start-reload.sh

networks:
Expand Down