Flyin Dockerfile

[Future-Outlier]

Tracking issue

flyteorg/flyte#4284

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Setup Process

Dockerfile

ARG PYTHON_VERSION
FROM python:${PYTHON_VERSION}-slim-buster
MAINTAINER Flyte Team <[email protected]>
LABEL org.opencontainers.image.source https://github.com/flyteorg/flytekit
WORKDIR /root
ENV PYTHONPATH /root

ARG VERSION
ARG TARGETARCH

# 1. Update the necessary packages for flytekit
# 2. Install code-server
# 3. Download code-server extensions for Python and Jupyter via wget
# 4. Install flytekit and flytekit-flyin with no cache
# 5. Delete apt cache. Reference: https://gist.github.com/marvell/7c812736565928e602c4
# 6. Some packages will create config file under /home by default, so we need to make sure it's writable
# 7. Change the permission of /tmp, so that others can run command on it
RUN apt-get update \
    && apt-get install build-essential git wget -y \
    && mkdir -p /tmp/ \
    && mkdir -p /tmp/code-server \
    && wget --no-check-certificate -O /tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}.tar.gz https://github.com/coder/code-server/releases/download/v4.19.0/code-server-4.19.0-linux-${TARGETARCH}.tar.gz \
    && tar -xzf /tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}.tar.gz -C /tmp/code-server/ \
    && wget --no-check-certificate https://open-vsx.org/api/ms-python/python/2023.20.0/file/ms-python.python-2023.20.0.vsix -P /tmp/code-server \
    && wget --no-check-certificate https://open-vsx.org/api/ms-toolsai/jupyter/2023.9.100/file/ms-toolsai.jupyter-2023.9.100.vsix -P /tmp/code-server \
    && pip install -U git+https://github.com/Future-Outlier/flytekit.git@7470baf933cd8b0da1bc27184ed6dfbb56a25dc2#subdirectory=plugins/flytekit-flyin \
    && apt-get clean autoclean \
    && apt-get autoremove --yes \
    && rm -rf /var/lib/{apt,dpkg,cache,log}/ \
    && useradd -u 1000 flytekit \
    && chown flytekit: /root \
    && chown flytekit: /home \
    && chown -R flytekit:flytekit /tmp \
    && chmod 755 /tmp   

# Set the environment variable for code-server
ENV PATH="/tmp/code-server/code-server-4.19.0-linux-${TARGETARCH}/bin:${PATH}"

USER flytekit

# Install extensions using code-server
# Execution is performed here as code-server configuration depends on the USER setting
# If we install it as ROOT, the config will be stored in /root/.config/code-server/config.yaml
# Now, the config of code-server will be stored in /home/flytekit/.config/code-server/config.yaml
RUN code-server --install-extension /tmp/code-server/ms-python.python-2023.20.0.vsix \
    && code-server --install-extension /tmp/code-server/ms-toolsai.jupyter-2023.9.100.vsix

For amd64

docker buildx build -t localhost:30000/amd64:amd . -f DockerfileFlyin --platform linux/amd64 --build-arg PYTHON_VERSION=3.9 --no-cache
docker run -it localhost:30000/amd64:amd /bin/bash
code-server --list-extensions

For arm64

docker buildx build -t localhost:30000/arm64:arm . -f DockerfileFlyin --platform linux/arm64 --build-arg PYTHON_VERSION=3.9 --no-cache
docker run -it localhost:30000/arm64:arm /bin/bash
code-server --list-extensions

Screenshots

Image

amd64

arm64

We skiped the code server download process.

We can use the extension by the dockerfile, not download in the runtime.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9c8f5d7) 86.15% compared to head (ac7d690) 86.15%.
Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1990   +/-   ##
=======================================
  Coverage   86.15%   86.15%           
=======================================
  Files         316      316           
  Lines       23330    23330           
  Branches     3457     3457           
=======================================
+ Hits        20099    20101    +2     
+ Misses       2639     2638    -1     
+ Partials      592      591    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ByronHsu]

Thanks!

[ByronHsu]

thanks for the amazing work! this can make using flyin very easy.

[eapolinario]

this should be TARGETARCH instead (as per the docker docs). We want the architecture that we're building this image for, not the arch of the node building it.

[Future-Outlier]

Yes, you are right!
Thank you really much!!!

[eapolinario]

we missed the python version here (like this). Do we need to have multiple versions of python in this image?

[eapolinario]

This is causing an issue in releases: https://github.com/flyteorg/flytekit/actions/runs/7108516439/workflow#L142

[Future-Outlier]

no problem, thanks a lot!