Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1pt] PR: Change Docker base image and stop Whitebox downloads #1291

Merged
merged 14 commits into from
Sep 25, 2024
37 changes: 12 additions & 25 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
## Temporary image to build the libraries and only save the needed artifacts
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.8.3 AS builder
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4 AS builder
WORKDIR /opt/builder
ARG dataDir=/data
ARG projectDir=/foss_fim
Expand All @@ -10,7 +10,7 @@ ENV taudemDir=$depDir/taudem/bin
ENV taudemDir2=$depDir/taudem_accelerated_flowDirections/taudem/build/bin

# remove reference to missing repo
RUN rm /etc/apt/sources.list.d/apache-arrow.sources
# RUN rm /etc/apt/sources.list.d/apache-arrow.sources

RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -45,7 +45,7 @@ RUN cd taudem_accelerated_flowDirections/taudem/build/bin && mv -t $taudemDir2 d

###############################################################################################
# Base Image that has GDAL, PROJ, etc
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.8.3
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4
ARG dataDir=/data
ENV projectDir=/foss_fim
ARG depDir=/dependencies
Expand All @@ -70,20 +70,12 @@ RUN mkdir -p $depDir
COPY --from=builder $depDir $depDir

# remove reference to missing repo
RUN rm /etc/apt/sources.list.d/apache-arrow.sources
# RUN rm /etc/apt/sources.list.d/apache-arrow.sources

RUN apt-get update --fix-missing && apt-get install -y openjdk-21-jdk && rm -rf /var/lib/apt/lists/*
RUN apt-get update --fix-missing && rm -rf /var/lib/apt/lists/*
RUN apt update --fix-missing

# An older version of openjdk still exists on the file system but was never cleaned up
# After research, we realized, it just needs file cleanup. Leaving it there is triggering security warnings
# RUN apt-get remove -y openjdk-17-jdk (not installed, just residue left)
RUN rm -rf ./usr/lib/jvm/*java-1.17* && \
rm -rf ./usr/lib/jvm/.java-1.17* && \
rm -rdf ./usr/lib/jvm/java-17*


RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt install -y p7zip-full python3-pip time mpich parallel libgeos-dev expect tmux rsync tzdata
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt install -y p7zip-full python3-pip time mpich parallel libgeos-dev expect tmux rsync tzdata wget

RUN apt auto-remove

Expand All @@ -99,18 +91,15 @@ ENV PYTHONUNBUFFERED=TRUE

## ADD TO PATHS ##
ENV PATH="$projectDir:${PATH}"
#ENV PATH=${PATH}:$projectDir:$projectDir/$srcDir:$projectDir/tools
# Jul 17, 2024: Even though PYTHONPATH isn't used, it still seems to want it.
ENV PYTHONPATH=${PATH}:$srcDir:$projectDir/tools
ENV PYTHONPATH="${PYTHONPATH}:${projectDir}:${srcDir}:${srcDir}/utils:${projectDir}/data:${toolsDir}"

## install python 3 modules ##

COPY Pipfile .
COPY Pipfile.lock .
RUN pip3 install pipenv==2024.0.1 && PIP_NO_CACHE_DIR=off pipenv install --system --deploy --ignore-pipfile

# ----------------------------------
# Mar 2023
# Mar 2023 / Sep 2024
# There are some nuances in the whitebox python downloads in that the first time it loads
# it goes to the internet and downloads the latest/greatest WBT (whiteboxtools) engine which is
# required for the whitebox python library to work. We don't want to have FIM attempting a download
Expand All @@ -120,12 +109,10 @@ RUN pip3 install pipenv==2024.0.1 && PIP_NO_CACHE_DIR=off pipenv install --syste
# We download and unzip it to the same file folder that pip deployed the whitebox library.
# Whitebox also attempts to always download a folder called testdata regardless of use.
# We added an empty folder to fake out whitebox_tools.py so it doesn't try to download the folder

# RUN wbox_path=/usr/local/lib/python3.10/dist-packages/whitebox/WBT && \
# wget -P $wbox_path https://www.whiteboxgeo.com/WBT_Linux/WhiteboxTools_linux_musl.zip && \
# unzip -o $wbox_path/WhiteboxTools_linux_musl.zip -d $wbox_path && \
# cp $wbox_path/whitebox_tools $wbox_path && \
# mkdir $wbox_path/testdata
ENV WBT_PATH=/usr/local/lib/python3.10/dist-packages/whitebox/WBT
RUN wget -P $WBT_PATH https://www.whiteboxgeo.com/WBT_Linux/WhiteboxTools_linux_musl.zip && \
unzip -o $WBT_PATH/WhiteboxTools_linux_musl.zip -d $WBT_PATH && \
cp $WBT_PATH/WhiteboxTools_linux_amd64/WBT/whitebox_tools $WBT_PATH
# ----------------------------------

## RUN UMASK TO CHANGE DEFAULT PERMISSIONS ##
Expand Down
6 changes: 3 additions & 3 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ certifi = "==2024.7.4"
dask = "==2024.6.2"
dask-expr = "==1.1.6"
distributed = "==2024.6.2"
fiona = "==1.9.6"
fiona = "==1.10.0"
flake8 = "==6.0.0"
flake8-pyproject = "==1.2.3"
geopandas = "==1.0.1"
gval = "==0.2.7.post1"
ipympl = "==0.9.3"
isort = "==5.12.0"
jupyter = "==1.0.0"
jupyterlab = "==3.6.7"
jupyterlab = "==3.6.8"
lmoments3 = "==1.0.6"
monaco = "==0.13.1"
natsort = "==8.3.1"
Expand Down Expand Up @@ -50,7 +50,7 @@ seaborn = "==0.12.2"
tables = "==3.8.0"
tqdm = "==4.66.3"
urllib3 = "==1.26.19"
whitebox = "==2.3.4"
whitebox = "==2.3.5"
xarray = "==2024.6.0"
zarr = "==2.18.0"

Expand Down
Loading
Loading