Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Hadolint to fail builds #735

Merged
merged 14 commits into from
Jan 24, 2025
Merged
12 changes: 6 additions & 6 deletions .github/workflows/docker-build-upload.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,6 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Run Hadolint
run: |
sudo curl -L https://github.com/hadolint/hadolint/releases/download/v${{ env.HADOLINT_VERSION }}/hadolint-Linux-x86_64 --output hadolint
sudo chmod +x hadolint
./hadolint images/${{ inputs.directory }}/Dockerfile --no-fail

- name: Echo disk usage before clean up
run: ./.github/scripts/echo_usage.sh

Expand Down Expand Up @@ -93,6 +87,12 @@ jobs:
run: |
sed -i '1i FROM ${{ env.BASE_IMAGE}} as ${{ inputs.image }}' ./images/${{ inputs.directory }}/Dockerfile

- name: Run Hadolint
run: |
sudo curl -L https://github.com/hadolint/hadolint/releases/download/v${{ env.HADOLINT_VERSION }}/hadolint-Linux-x86_64 --output hadolint
sudo chmod +x hadolint
./hadolint images/${{ inputs.directory }}/Dockerfile --failure-threshold error

# make build emits full_image_name, image_tag, and image_repo outputs
- name: Build image
id: build-image
Expand Down
56 changes: 28 additions & 28 deletions images/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ RUN mamba install --quiet \
's3fs' \
'fire==0.5.0' \
'graphviz' && \
pip install 'kubeflow-training' && \
pip install --no-cache-dir 'kubeflow-training' && \
clean-layer.sh && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER && \
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER" && \
chmod +x /usr/local/bin/aaw-suspend-server.sh

# Add helpers for shell initialization
Expand Down Expand Up @@ -74,15 +74,15 @@ RUN \
apt-get update && \
apt-get install -y openjdk-8-jre && \
apt-get clean && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER \
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER" \
&& \
# kubectl
curl -LO "${KUBECTL_URL}" \
&& curl -LO "${KUBECTL_URL}.sha256" \
&& echo "$(cat kubectl.sha256) kubectl" | sha256sum -c - \
&& chmod +x ./kubectl \
&& sudo mv ./kubectl /usr/local/bin/kubectl \
&& mv ./kubectl /usr/local/bin/kubectl \
&& \
# AzureCLI - installation script from Azure
curl -sLO "${AZCLI_URL}" \
Expand All @@ -97,36 +97,36 @@ RUN \
&& \
# trino cli
wget -q "${TRINO_URL}" -O /tmp/trino-original \
&& echo ${TRINO_SHA} /tmp/trino-original | sha256sum -c \
&& echo "${TRINO_SHA}" /tmp/trino-original | sha256sum -c \
&& echo "trinocli: ok" \
&& chmod +x /tmp/trino-original \
&& sudo mv /tmp/trino-original /usr/local/bin/trino-original \
&& mv /tmp/trino-original /usr/local/bin/trino-original \
&& \
# argo cli
curl -sLO ${ARGO_CLI_URL}\
&& curl -LO ${ARGO_CLI_CHECKSUM_URL} \
curl -sLO "${ARGO_CLI_URL}" \
&& curl -LO "${ARGO_CLI_CHECKSUM_URL}" \
&& grep argo-linux-amd64.gz argo-workflows-cli-checksums.txt | sha256sum -c - \
&& gunzip argo-linux-amd64.gz \
&& chmod +x argo-linux-amd64 \
&& sudo mv ./argo-linux-amd64 /usr/local/bin/argo \
&& mv ./argo-linux-amd64 /usr/local/bin/argo \
&& argo version \
&& \
# quarto
curl -sLO ${QUARTO_URL} \
&& curl -LO ${QUARTO_CHECKSUM_URL} \
&& grep "quarto-${QUARTO_VERSION}-linux-amd64.tar.gz" quarto-${QUARTO_VERSION}-checksums.txt | sha256sum -c - \
&& tar -xf quarto-${QUARTO_VERSION}-linux-amd64.tar.gz \
&& chmod +x quarto-${QUARTO_VERSION} \
&& sudo rm -f /usr/local/bin/quarto \
&& sudo mv ./quarto-${QUARTO_VERSION} /usr/local/bin/quarto
curl -sLO "${QUARTO_URL}" \
&& curl -LO "${QUARTO_CHECKSUM_URL}" \
&& grep "quarto-${QUARTO_VERSION}-linux-amd64.tar.gz" "quarto-${QUARTO_VERSION}-checksums.txt" | sha256sum -c - \
&& tar -xf "quarto-${QUARTO_VERSION}-linux-amd64.tar.gz" \
&& chmod +x "quarto-${QUARTO_VERSION}" \
&& rm -f /usr/local/bin/quarto \
&& mv "./quarto-${QUARTO_VERSION}" /usr/local/bin/quarto
EveningStarlight marked this conversation as resolved.
Show resolved Hide resolved

# ODBC drivers
RUN apt-get update && \
apt-get install -y unixodbc-dev && \
pip install --no-cache-dir --quiet pyodbc && \
rm -rf /var/lib/apt/lists/* && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER"

# Forcibly upgrade packages to patch vulnerabilities
# See https://github.com/StatCan/aaw-private/issues/58#issuecomment-1471863092 for more details.
Expand Down Expand Up @@ -182,7 +182,7 @@ COPY vscode-overrides.json $CS_TEMP_HOME/Machine/settings.json
# Languagepacks.json needs to exist for code-server to recognize the languagepack
COPY languagepacks.json $CS_TEMP_HOME/

RUN pip install \
RUN pip install --no-cache-dir \
'git+https://github.com/betatim/vscode-binder' && \
# jupyter_contrib_nbextensions likes to be installed with pip
mamba install --quiet --yes -c conda-forge \
Expand All @@ -209,10 +209,10 @@ RUN pip install \
jupyter lab build && \
jupyter lab clean && \
clean-layer.sh && \
rm -rf /home/$NB_USER/.cache/yarn && \
rm -rf /home/$NB_USER/.node-gyp && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER
rm -rf "/home/$NB_USER/.cache/yarn" && \
rm -rf "/home/$NB_USER/.node-gyp" && \
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER"

# Update and pin packages
# See https://github.com/StatCan/aaw-kubeflow-containers/issues/293
Expand All @@ -232,8 +232,8 @@ RUN julia -e 'using Pkg; Pkg.add("LanguageServer")' && \
'unified-language-server' \
'yaml-language-server' && \
clean-layer.sh && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER"

# OpenM install
# Install OpenM++ MPI
Expand Down Expand Up @@ -267,7 +267,7 @@ RUN apt-get update --yes \
&& rm -r ${OMPP_INSTALL_DIR}/html \
&& mv ${OMPP_INSTALL_DIR}/ompp-ui/dist/spa ${OMPP_INSTALL_DIR}/html \
&& fix-permissions ${OMPP_INSTALL_DIR} \
&& pip install /opt/jupyter-ompp-proxy/
&& pip install --no-cache-dir /opt/jupyter-ompp-proxy/

# Solarized Theme and Cell Execution Time
COPY jupyterlab-overrides.json /opt/conda/share/jupyter/lab/settings/overrides.json
Expand Down
2 changes: 1 addition & 1 deletion images/cmd/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ RUN conda config --add channels http://jfrog-platform-artifactory.jfrog-system:8
conda config --add channels http://jfrog-platform-artifactory.jfrog-system:8081/artifactory/api/conda/conda-pytorch-remote --system

# Assign ownership of user's home directory to user
RUN chown $NB_USER:users /home/$NB_USER
RUN chown "$NB_USER:users" "/home/$NB_USER"

USER $NB_USER
ENTRYPOINT ["tini", "--"]
Expand Down
20 changes: 10 additions & 10 deletions images/platform/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,22 @@ USER root
ARG AOCL_VERSION=4.0
ENV AOCL_PATH=/opt/amd/aocl/${AOCL_VERSION}
ARG AOCL_SHA256=8a249e727beb8005639b4887074e1ea75020267ed1ac25520876a7ad21d0f4f6
RUN cd ${RESOURCES_PATH} && \
wget --quiet https://download.amd.com/developer/eula/aocl/aocl-4-0/aocl-linux-aocc-${AOCL_VERSION}.tar.gz -O /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar && \
WORKDIR ${RESOURCES_PATH}
RUN \
wget --quiet "https://download.amd.com/developer/eula/aocl/aocl-4-0/aocl-linux-aocc-${AOCL_VERSION}.tar.gz" -O "/tmp/aocl-linux-aocc-${AOCL_VERSION}.tar" && \
echo "${AOCL_SHA256} /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar" | sha256sum -c - && \
tar xf /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar -C ./ && \
cd ./aocl-linux-aocc-${AOCL_VERSION} && \
/bin/bash ./install.sh -t /opt/amd/aocl && \
cp setenv_aocl.sh ${AOCL_PATH} &&\
rm /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar
tar xf "/tmp/aocl-linux-aocc-${AOCL_VERSION}.tar" -C ./ && \
/bin/bash "./aocl-linux-aocc-${AOCL_VERSION}/install.sh" -t /opt/amd/aocl && \
cp "./aocl-linux-aocc-${AOCL_VERSION}/setenv_aocl.sh" ${AOCL_PATH} &&\
rm "/tmp/aocl-linux-aocc-${AOCL_VERSION}.tar"

# Install AMD AOCC
ARG AOCC_VERSION=4.0.0
ARG AOCC_SHA256=2729ec524cbc927618e479994330eeb72df5947e90cfcc49434009eee29bf7d4
RUN cd ${RESOURCES_PATH} && \
WORKDIR ${RESOURCES_PATH}
RUN \
wget --quiet https://download.amd.com/developer/eula/aocc-compiler/aocc-compiler-${AOCC_VERSION}.tar -O /tmp/aocc-compiler-${AOCC_VERSION}.tar && \
echo "${AOCC_SHA256} /tmp/aocc-compiler-${AOCC_VERSION}.tar" | sha256sum -c - && \
tar xf /tmp/aocc-compiler-${AOCC_VERSION}.tar -C ./ && \
cd ./aocc-compiler-${AOCC_VERSION} && \
/bin/bash ./install.sh && \
/bin/bash ./aocc-compiler-${AOCC_VERSION}/install.sh && \
rm /tmp/aocc-compiler-${AOCC_VERSION}.tar
4 changes: 2 additions & 2 deletions images/pytorch/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ RUN mamba create -n torch && \
# pytorch-cuda are the nvidia cuda drivers
pytorch-cuda=11.8 && \
clean-layer.sh && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER && \
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER" && \
python -m ipykernel install --user --name torch --display-name "PyTorch"

66 changes: 32 additions & 34 deletions images/remote-desktop/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ RUN apt-get update --yes \
&& rm -rf /var/lib/apt/lists/*

RUN /rocker_scripts/install_shiny_server.sh \
&& pip3 install jupyter \
&& pip3 install --no-cache-dir jupyter \
&& rm -rf /var/lib/apt/lists/*

# Users should install R packages in their home directory
Expand Down Expand Up @@ -78,10 +78,10 @@ RUN mamba install --quiet \
's3fs' \
'fire==0.5.0' \
'graphviz' && \
pip install 'kubeflow-training' && \
pip install --no-cache-dir 'kubeflow-training' && \
clean-layer.sh && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER && \
fix-permissions "$CONDA_DIR" && \
fix-permissions "/home/$NB_USER" && \
chmod +x /usr/local/bin/aaw-suspend-server.sh

# Add helpers for shell initialization
Expand Down Expand Up @@ -138,7 +138,7 @@ RUN \
&& curl -LO "${KUBECTL_URL}.sha256" \
&& echo "$(cat kubectl.sha256) kubectl" | sha256sum -c - \
&& chmod +x ./kubectl \
&& sudo mv ./kubectl /usr/local/bin/kubectl \
&& mv ./kubectl /usr/local/bin/kubectl \
&& \
# AzureCLI - installation script from Azure
curl -sLO "${AZCLI_URL}" \
Expand All @@ -156,15 +156,15 @@ RUN \
&& echo ${TRINO_SHA} /tmp/trino-original | sha256sum -c \
&& echo "trinocli: ok" \
&& chmod +x /tmp/trino-original \
&& sudo mv /tmp/trino-original /usr/local/bin/trino-original \
&& mv /tmp/trino-original /usr/local/bin/trino-original \
&& \
# argo cli
curl -sLO ${ARGO_CLI_URL}\
&& curl -LO ${ARGO_CLI_CHECKSUM_URL} \
&& grep argo-linux-amd64.gz argo-workflows-cli-checksums.txt | sha256sum -c - \
&& gunzip argo-linux-amd64.gz \
&& chmod +x argo-linux-amd64 \
&& sudo mv ./argo-linux-amd64 /usr/local/bin/argo \
&& mv ./argo-linux-amd64 /usr/local/bin/argo \
&& argo version \
&& \
# quarto
Expand All @@ -173,8 +173,8 @@ RUN \
&& grep "quarto-${QUARTO_VERSION}-linux-amd64.tar.gz" quarto-${QUARTO_VERSION}-checksums.txt | sha256sum -c - \
&& tar -xf quarto-${QUARTO_VERSION}-linux-amd64.tar.gz \
&& chmod +x quarto-${QUARTO_VERSION} \
&& sudo rm -f /usr/local/bin/quarto \
&& sudo mv ./quarto-${QUARTO_VERSION} /usr/local/bin/quarto
&& rm -f /usr/local/bin/quarto \
&& mv ./quarto-${QUARTO_VERSION} /usr/local/bin/quarto

ENV NB_UID=1000
ENV NB_GID=100
Expand Down Expand Up @@ -356,7 +356,7 @@ RUN \
# Cleanup
clean-layer.sh

RUN pip3 install --quiet 'selenium' && \
RUN pip3 install --quiet --no-cache-dir 'selenium' && \
fix-permissions $CONDA_DIR && \
fix-permissions /home/$NB_USER

Expand Down Expand Up @@ -393,12 +393,12 @@ RUN apt-get update --yes \
# https://github.com/cdr/code-server/issues/171
ARG SHA256py=a4191fefc0e027fbafcd87134ac89a8b1afef4fd8b9dc35f14d6ee7bdf186348
ARG SHA256gl=ed130b2a0ddabe5132b09978195cefe9955a944766a72772c346359d65f263cc

RUN cd $RESOURCES_PATH \
&& mkdir -p $HOME/.local/share \
&& mkdir -p $VSCODE_DIR/extensions
ENV VS_PYTHON_VERSION="2020.5.86806"
RUN wget --quiet --no-check-certificate https://github.com/microsoft/vscode-python/releases/download/$VS_PYTHON_VERSION/ms-python-release.vsix \

WORKDIR ${RESOURCES_PATH}
RUN mkdir -p $HOME/.local/share \
&& mkdir -p $VSCODE_DIR/extensions \
&& wget --quiet --no-check-certificate https://github.com/microsoft/vscode-python/releases/download/$VS_PYTHON_VERSION/ms-python-release.vsix \
&& echo "${SHA256py} ms-python-release.vsix" | sha256sum -c - \
&& bsdtar -xf ms-python-release.vsix extension \
&& rm ms-python-release.vsix \
Expand All @@ -418,17 +418,17 @@ RUN git clone -vb release/$VS_LOCALE_REPO_VERSION https://github.com/microsoft/v
WORKDIR $NODE_VERSION_ARCH

RUN if [ -f /usr/bin/node ]; then \
sudo mv -f /usr/bin/node /usr/bin/node.old; \
mv -f /usr/bin/node /usr/bin/node.old; \
fi \
&& if [ -f /usr/bin/npm ]; then \
sudo mv -f /usr/bin/npm /usr/bin/npm.old; \
mv -f /usr/bin/npm /usr/bin/npm.old; \
fi \
&& if [ -f /usr/bin/npx ]; then \
sudo mv -f /usr/bin/npx /usr/bin/npx.old; \
mv -f /usr/bin/npx /usr/bin/npx.old; \
fi \
&& sudo ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/node /usr/bin/node \
&& sudo ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/npm /usr/bin/npm \
&& sudo ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/npx /usr/bin/npx \
&& ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/node /usr/bin/node \
&& ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/npm /usr/bin/npm \
&& ln -sfv /tmp/vscode-lang-pack-install/$NODE_VERSION_ARCH/bin/npx /usr/bin/npx \
&& npm install @vscode/vsce

WORKDIR /tmp/vscode-lang-pack-install/vscode-loc/i18n/vscode-language-pack-fr
Expand All @@ -442,13 +442,13 @@ WORKDIR /tmp

RUN rm -fr vscode-lang-pack-install \
&& if [ -f /usr/bin/node.old ]; then \
sudo mv -f /usr/bin/node.old /usr/bin/node; \
mv -f /usr/bin/node.old /usr/bin/node; \
fi \
&& if [ -f /usr/bin/npm.old ]; then \
sudo mv -f /usr/bin/npm.old /usr/bin/npm; \
mv -f /usr/bin/npm.old /usr/bin/npm; \
fi \
&& if [ -f /usr/bin/npx.old ]; then \
sudo mv -f /usr/bin/npx.old /usr/bin/npx; \
mv -f /usr/bin/npx.old /usr/bin/npx; \
fi \
&& fix-permissions $XDG_DATA_HOME \
&& clean-layer.sh
Expand Down Expand Up @@ -510,8 +510,8 @@ COPY French/vscode/languagepacks.json /home/$NB_USER/.config/Code/

#Tiger VNC
ARG SHA256tigervnc=fb8f94a5a1d77de95ec8fccac26cb9eaa9f9446c664734c68efdffa577f96a31
WORKDIR ${RESOURCES_PATH}
RUN \
cd ${RESOURCES_PATH} && \
wget --quiet https://sourceforge.net/projects/tigervnc/files/stable/1.10.1/tigervnc-1.10.1.x86_64.tar.gz/ -O /tmp/tigervnc.tar.gz && \
echo "${SHA256tigervnc} /tmp/tigervnc.tar.gz" | sha256sum -c - && \
tar xzf /tmp/tigervnc.tar.gz --strip 1 -C / && \
Expand All @@ -536,7 +536,6 @@ RUN rm /usr/share/applications/exo-mail-reader.desktop
#Prevent screen from locking
RUN apt-get remove -y -q light-locker


# apt-get may result in root-owned directories/files under $HOME
RUN usermod -l $NB_USER rstudio && \
chown -R $NB_UID:$NB_GID $HOME
Expand All @@ -548,7 +547,6 @@ RUN apt-get update && apt-get install --yes websockify \
&& cp /usr/lib/websockify/rebind.cpython-38-x86_64-linux-gnu.so /usr/lib/websockify/rebind.so \
&& clean-layer.sh


#Set Defaults
ENV HOME=/home/$NB_USER
COPY /novnc $RESOURCES_PATH/novnc
Expand Down Expand Up @@ -595,24 +593,24 @@ RUN pip3 --no-cache-dir install --quiet \
ARG AOCL_VERSION=4.0
ENV AOCL_PATH=/opt/amd/aocl/${AOCL_VERSION}
ARG AOCL_SHA256=8a249e727beb8005639b4887074e1ea75020267ed1ac25520876a7ad21d0f4f6
RUN cd ${RESOURCES_PATH} && \
WORKDIR ${RESOURCES_PATH}
RUN \
wget --quiet https://download.amd.com/developer/eula/aocl/aocl-4-0/aocl-linux-aocc-${AOCL_VERSION}.tar.gz -O /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar && \
echo "${AOCL_SHA256} /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar" | sha256sum -c - && \
tar xf /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar -C ./ && \
cd ./aocl-linux-aocc-${AOCL_VERSION} && \
/bin/bash ./install.sh -t /opt/amd/aocl && \
cp setenv_aocl.sh ${AOCL_PATH} &&\
/bin/bash ./aocl-linux-aocc-${AOCL_VERSION}/install.sh -t /opt/amd/aocl && \
cp ./aocl-linux-aocc-${AOCL_VERSION}/setenv_aocl.sh ${AOCL_PATH} &&\
rm /tmp/aocl-linux-aocc-${AOCL_VERSION}.tar

# Install AMD AOCC
ARG AOCC_VERSION=4.0.0
ARG AOCC_SHA256=2729ec524cbc927618e479994330eeb72df5947e90cfcc49434009eee29bf7d4
RUN cd ${RESOURCES_PATH} && \
WORKDIR ${RESOURCES_PATH}
RUN \
wget --quiet https://download.amd.com/developer/eula/aocc-compiler/aocc-compiler-${AOCC_VERSION}.tar -O /tmp/aocc-compiler-${AOCC_VERSION}.tar && \
echo "${AOCC_SHA256} /tmp/aocc-compiler-${AOCC_VERSION}.tar" | sha256sum -c - && \
tar xf /tmp/aocc-compiler-${AOCC_VERSION}.tar -C ./ && \
cd ./aocc-compiler-${AOCC_VERSION} && \
/bin/bash ./install.sh && \
/bin/bash ./aocc-compiler-${AOCC_VERSION}/install.sh && \
rm /tmp/aocc-compiler-${AOCC_VERSION}.tar


Expand Down
Loading
Loading