Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding osm_land provider and Map-styles updates #489

Merged
merged 7 commits into from
Mar 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions hetzner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ Ensure you are using the correct Docker images for production deployment, https:

```sh
docker compose -f hetzner/tiler.production.yml up -d
# docker compose -f hetzner/tiler.production.yml up tiler_production -d
# docker compose -f hetzner/tiler.production.yml up imposm_production -d
# docker compose -f hetzner/tiler.production.yml up cache_production -d
# docker compose -f hetzner/tiler.production.yml up global_seeding_production -d
# docker compose -f hetzner/tiler.production.yml up tile_coverage_seeding_production -d

```

🛠 Deploying to Staging
Expand All @@ -30,3 +36,4 @@ docker compose -f hetzner/tiler.staging.yml up tile_coverage_seeding -d
• Ensure that you are using the correct Docker images for each environment.
• Manually update the images before deploying production services.
• For troubleshooting, check logs using:

68 changes: 68 additions & 0 deletions hetzner/seed.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/bin/bash

seed_global() {
while true; do
echo "Starting global seeding..."
pkill -f "tegola" && sleep 5

# Seed all areas (min 0 to max 5)
tegola cache seed tile-name "0/0/0" \
--config=/opt/tegola_config/config.toml \
--map=osm \
--min-zoom=0 \
--max-zoom=5 \
--concurrency=32 \
--overwrite=true

# Seed land areas separately (zoom 6-7)
for zoom in $(seq 6 7); do
echo "Downloading tile list for zoom level $zoom..."
wget -O /opt/tile-list.tiles "https://s3.amazonaws.com/planet.openhistoricalmap.org/tile_coverage/tiles_boundary_$zoom.list"

tegola cache seed tile-list /opt/tile-list.tiles \
--config=/opt/tegola_config/config.toml \
--map=osm \
--min-zoom="$zoom" \
--max-zoom="$zoom" \
--concurrency=32 \
--overwrite=true
done
echo "Global seeding completed. Sleeping for 1 hour..."
sleep 3600
done
}

seed_coverage() {
while true; do
echo "Starting coverage seeding..."
wget -O /opt/tile-list.tiles "https://s3.amazonaws.com/planet.openhistoricalmap.org/tile_coverage/tiles_14.list"
pkill -f "tegola" && sleep 5

tegola cache seed tile-list /opt/tile-list.tiles \
--config=/opt/tegola_config/config.toml \
--map=osm \
--min-zoom=0 \
--max-zoom=14 \
--concurrency=32 \
--overwrite=false
sleep 300
done
}

if [ "$#" -ne 1 ]; then
echo "Usage: $0 <global|coverage>"
exit 1
fi

case "$1" in
global)
seed_global
;;
coverage)
seed_coverage
;;
*)
echo "Invalid option. Use 'global' or 'coverage'."
exit 1
;;
esac
34 changes: 7 additions & 27 deletions hetzner/tiler.production.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ services:
- tiler_network
# Tiler server
tiler_production:
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2241.hba0f0a5
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2248.he042bfb
ports:
- "9090:9090"
env_file:
Expand All @@ -75,23 +75,13 @@ services:
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2234.hfa6976e
env_file:
- .env.production
volumes:
- ./seed.sh:/opt/seed.sh
entrypoint:
- /bin/bash
- "-c"
- |
set -x
while true; do
pkill -f "tegola" && sleep 5
echo "0/0/0" > /opt/tile-list.tiles
tegola cache seed tile-list /opt/tile-list.tiles \
--config=/opt/tegola_config/config.toml \
--map=osm \
--min-zoom=0 \
--max-zoom=7\
--concurrency=32 \
--overwrite=true
sleep 3600
done
/opt/seed.sh global
deploy:
resources:
limits:
Expand All @@ -104,23 +94,13 @@ services:
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2234.hfa6976e
env_file:
- .env.production
volumes:
- ./seed.sh:/opt/seed.sh
entrypoint:
- /bin/bash
- "-c"
- |
set -x
while true; do
wget -O tile-list.tiles https://s3.amazonaws.com/planet.openhistoricalmap.org/tile_coverage/tiles.list
pkill -f "tegola" && sleep 5
tegola cache seed tile-list tile-list.tiles \
--config=/opt/tegola_config/config.toml \
--map=osm \
--min-zoom=8 \
--max-zoom=14 \
--concurrency=32 \
--overwrite=false
sleep 3600
done
/opt/seed.sh coverage
deploy:
resources:
limits:
Expand Down
2 changes: 1 addition & 1 deletion hetzner/tiler.staging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ services:
cpus: '1'
memory: 1G
tiler:
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2234.hfa6976e
image: ghcr.io/openhistoricalmap/tiler-server:0.0.1-0.dev.git.2248.he042bfb
ports:
- "9091:9090"
env_file:
Expand Down
3 changes: 1 addition & 2 deletions images/tiler-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@ RUN mkdir /opt/tegola_config
RUN cd /opt/ && python build_config.py \
--output=/opt/tegola_config/config.toml \
--provider_names \
"land,\
admin_boundaries_lines,\
"admin_boundaries_lines,\
admin_boundaries.centroids,\
admin_boundaries_maritime,\
place_areas,\
Expand Down
92 changes: 0 additions & 92 deletions images/tiler-server/config/providers/land.toml

This file was deleted.

2 changes: 1 addition & 1 deletion images/web/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ RUN apache2ctl configtest
RUN chown -R www-data: $workdir

# Clone Map-styles
ENV OPENHISTORICALMAP_MAP_STYLES_GITSHA=6de6e329569ab6561d1fa6ab2cebb42df7fd17c7
ENV OPENHISTORICALMAP_MAP_STYLES_GITSHA=7a70bbef491db6928f28256a3a68d3158a6f9710
RUN git clone --branch staging --depth 1 https://github.com/OpenHistoricalMap/map-styles.git $workdir/public/map-styles
RUN cd $workdir/public/map-styles && git fetch --depth 1 origin $OPENHISTORICALMAP_MAP_STYLES_GITSHA && git checkout $OPENHISTORICALMAP_MAP_STYLES_GITSHA
RUN rm -rf $workdir/public/map-styles/.git
Expand Down
2 changes: 1 addition & 1 deletion values.production.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -983,7 +983,7 @@ ohm:
label_value: web_large

tilerCacheSeedCoverage:
enabled: true
enabled: false
schedule: '0 * * * *'
env:
CONCURRENCY: 128
Expand Down
6 changes: 3 additions & 3 deletions values.staging.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ osm-seed:
nodeSelector:
enabled: true
label_key: nodegroup_type
label_value: db_large
label_value: web_medium
env:
POSTGRES_DB: {{STAGING_DB}}
POSTGRES_USER: {{STAGING_DB_USER}}
Expand Down Expand Up @@ -137,7 +137,7 @@ osm-seed:
nodeSelector:
enabled: true
label_key: nodegroup_type
label_value: db_large
label_value: web_medium
replicaCount: 1
# Set staticIp, if you are using cloudProvider=gcp
staticIp: c
Expand Down Expand Up @@ -249,7 +249,7 @@ osm-seed:
nodeSelector:
enabled: true
label_key: nodegroup_type
label_value: db_large
label_value: web_medium
env:
ENABLE_SEND_SLACK_MESSAGE: "true"
SLACK_WEBHOOK_URL: {{OHM_SLACK_WEBHOOK_URL}}
Expand Down
Loading