Skip to content

Remove helix script install (partial set of Dockerfiles) #1336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

Remove helix script install (partial set of Dockerfiles) #1336

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
65a76f2
Remove helix script install
mthalman Jan 30, 2025
075da16
Add back virtual environments
mthalman Feb 3, 2025
297153d
update python version in Windows dockerfiles
mthalman Feb 3, 2025
ee84975
update Windows Dockerfiles to install latest Python version
mthalman Feb 3, 2025
9e2980f
Update python comment for AlmaLinux
mthalman Feb 4, 2025
f48f7ff
Updates to fix issues with Windows Dockerfiles
mthalman Feb 6, 2025
78d5b6a
Create virtual env as helixbot user
mthalman Feb 11, 2025
b8b75aa
updates to Windows dockerfiles
mthalman Feb 11, 2025
01fd064
Fix setting of env vars
mthalman Feb 11, 2025
52f4b8e
Remove virtualenv installation
mthalman Feb 12, 2025
a36b6ec
Forgot to remove this
mthalman Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions src/almalinux/8/helix/amd64/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ RUN dnf upgrade --refresh -y \
# libmsquic except in the 8.1-specific packages feed.)
&& dnf config-manager --add-repo=https://packages.microsoft.com/centos/8/prod/config.repo \
&& dnf install --setopt tsflags=nodocs -y \
# Get recent python3 This is needed to get a pip recent enough
# not to fail the build when installing cryptography library as
# a dependency of the helix scripts.
# Install supported version of Python (default for python3 is 3.6 which is unsupported)
python39 \
# Required for asp.net core test runs
sudo \
Expand All @@ -21,9 +19,6 @@ RUN dnf upgrade --refresh -y \
file \
&& dnf clean all

RUN python3 -m pip download --no-deps helix-scripts --index-url https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple \
&& python3 -m pip install ./helix_scripts-*-py3-none-any.whl

# Aspnetcore test runs expect python to be available without version suffix
RUN alternatives --set python /usr/bin/python3 \
&& ln -sf /usr/bin/pip3 /usr/bin/pip
Expand All @@ -35,6 +30,9 @@ RUN adduser --uid 1000 --shell /bin/bash --gid adm helixbot \
&& chmod 755 /root \
&& echo "helixbot ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers

# Setup helix virtual environment
ENV VIRTUAL_ENV=/home/helixbot/.vsts-env
USER helixbot

RUN python3 -m venv /home/helixbot/.vsts-env
RUN python3 -m venv $VIRTUAL_ENV
USER root
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
25 changes: 3 additions & 22 deletions src/fedora/41/helix/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,3 @@
FROM library/fedora:41 AS venv

RUN dnf upgrade --refresh -y \
&& dnf install --setopt tsflags=nodocs -y \
dnf-plugins-core \
&& dnf install --setopt=install_weak_deps=False --setopt tsflags=nodocs -y \
openssl \
python3-devel \
python3-pip \
gcc \
libatomic \
redhat-rpm-config \
&& dnf clean all

RUN python3 -m venv /venv \
&& source /venv/bin/activate \
pip download --no-deps helix-scripts --index-url https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple \
&& pip download --no-deps helix-scripts --index-url https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple \
&& pip install ./helix_scripts-*-py3-none-any.whl

FROM library/fedora:41

# Install Dependencies
Expand Down Expand Up @@ -65,8 +45,9 @@ RUN /usr/sbin/adduser --uid 1000 --shell /bin/bash --group adm helixbot \
&& echo "helixbot ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers \
&& chmod +s /usr/bin/ping

# Install Helix Dependencies
# Setup helix virtual environment
ENV VIRTUAL_ENV=/home/helixbot/.vsts-env
USER helixbot
RUN python3 -m venv $VIRTUAL_ENV
USER root
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
COPY --from=venv --chown=helixbot /venv $VIRTUAL_ENV
39 changes: 19 additions & 20 deletions src/nanoserver/1809/helix/amd64/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,31 @@ RUN $apiUrl = 'https://api.github.com/repos/PowerShell/PowerShell/releases/lates
Expand-Archive -Path C:\powershell.zip -DestinationPath C:\powershell; `
Remove-Item -Path C:\powershell.zip

# Download latest stable version of Python
RUN $apiUrl = 'https://api.nuget.org/v3-flatcontainer/python/index.json'; `
$response = Invoke-RestMethod -Uri $apiUrl; `
$versions = $response.versions | Where-Object { $_ -notmatch '-' } | Sort-Object { [version]$_ } -Descending; `
$latestVersion = $versions[0]; `
echo \"Downloading Python $latestVersion\"; `
Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/python/$latestVersion -OutFile $env:TEMP\python.zip; `
md C:\PythonTemp; `
tar -zxf $env:TEMP\python.zip -C C:\PythonTemp;


FROM mcr.microsoft.com/windows/nanoserver:1809

SHELL ["cmd", "/S", "/C"]
USER ContainerAdministrator
ENTRYPOINT C:\Windows\System32\cmd.exe

COPY --from=installer [ "C:\\powershell\\", "C:\\Program Files\\PowerShell\\" ]

RUN curl -SL --output %TEMP%\python.zip https://www.nuget.org/api/v2/package/python/3.7.3 `
&& md C:\Python C:\PythonTemp `
&& tar -zxf %TEMP%\python.zip -C C:\PythonTemp `
&& xcopy /s c:\PythonTemp\tools C:\Python `
&& rd /s /q c:\PythonTemp `
&& del /q %TEMP%\python.zip `
&& setx /M PYTHONPATH "C:\Python\Lib;C:\Python\DLLs;"

ENV PATH="$PATH;C:\Program Files\PowerShell\;C:\Python;C:\python\scripts"

RUN md c:\\helixtmp && pushd c:\\helixtmp &&`
python -m pip install --upgrade pip==20.2 && `
python -m pip install virtualenv==16.6.0 && `
pip download --no-deps helix-scripts --index-url https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && `
for %f in (.\helix_scripts-*-py3-none-any.whl) do (pip install %f) && `
popd && rd /s /q c:\\helixtmp && `
COPY --from=installer [ "C:\\PythonTemp\\tools", "C:\\Python\\" ]

RUN setx /M PATH "%PATH%;C:\Program Files\PowerShell\;C:\Python;C:\python\scripts" && `
setx /M PYTHONPATH "C:\Python\Lib;C:\Python\DLLs;" && `
setx /M VIRTUAL_ENV "C:\Python-env"

RUN python -m pip install --upgrade pip && `
python -m venv %VIRTUAL_ENV% && `
pwsh -Command `
New-Item -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\' -Name 'dotnet.exe' -Force -ErrorAction SilentlyContinue ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\dotnet.exe' -Value 2 -Name DumpType -Force ; `
Expand All @@ -45,5 +45,4 @@ RUN md c:\\helixtmp && pushd c:\\helixtmp &&`
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 'C:\cores' -Name DumpFolder -Force ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 2 -Name DumpCount -Force

WORKDIR C:\\Work

WORKDIR C:\Work
42 changes: 23 additions & 19 deletions src/windowsservercore/ltsc2019/helix/amd64/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,29 @@
# escape=`
FROM mcr.microsoft.com/windows/servercore:ltsc2019

SHELL ["cmd", "/S", "/C"]
USER ContainerAdministrator
# Install latest stable version of Python
RUN powershell -Command "`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no big deal but this is easier to grok than the nanoserver approach using SHELL changes

$ErrorActionPreference = 'Stop'; `
$ProgressPreference = 'SilentlyContinue'; `
$apiUrl = 'https://api.nuget.org/v3-flatcontainer/python/index.json'; `
$response = Invoke-RestMethod -Uri $apiUrl; `
$versions = $response.versions | Where-Object { $_ -notmatch '-' } | Sort-Object { [version]$_ } -Descending; `
$latestVersion = $versions[0]; `
echo \"Downloading Python $latestVersion\"; `
Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/python/$latestVersion -OutFile $env:TEMP\python.zip; `
md C:\Python; `
md C:\PythonTemp; `
tar -zxf $env:TEMP\python.zip -C C:\PythonTemp; `
xcopy /s c:\PythonTemp\tools C:\Python; `
Remove-Item -Recurse -Force C:\PythonTemp; `
Remove-Item -Force $env:TEMP\python.zip;"

RUN curl -SL --output %TEMP%\python.zip https://www.nuget.org/api/v2/package/python/3.7.3 `
&& md C:\Python C:\PythonTemp `
&& tar -zxf %TEMP%\python.zip -C C:\PythonTemp `
&& xcopy /s c:\PythonTemp\tools C:\Python `
&& rd /s /q c:\PythonTemp `
&& del /q %TEMP%\python.zip `
&& setx /M PYTHONPATH "C:\Python\Lib;C:\Python\DLLs;"
RUN setx /M PATH "%PATH%;C:\Python;C:\python\scripts" && `
setx /M PYTHONPATH "C:\Python\Lib;C:\Python\DLLs;" && `
setx /M VIRTUAL_ENV "C:\Python-env"

RUN md c:\\helixtmp && pushd c:\\helixtmp &&`
C:\Python\python.exe -m pip install --upgrade pip==20.2 --no-warn-script-location && `
C:\Python\python.exe -m pip install virtualenv==16.6.0 --no-warn-script-location && `
C:\Python\python.exe -m pip download --no-deps helix-scripts --index-url https://dnceng.pkgs.visualstudio.com/public/_packaging/helix-client-prod/pypi/simple && `
for %f in (.\helix_scripts-*-py3-none-any.whl) do (C:\Python\python.exe -m pip install %f --no-warn-script-location) && `
popd && rd /s /q c:\\helixtmp && `
RUN python -m pip install --upgrade pip && `
python -m venv %VIRTUAL_ENV% && `
powershell -Command `
New-Item -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\' -Name 'dotnet.exe' -Force -ErrorAction SilentlyContinue ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\dotnet.exe' -Value 2 -Name DumpType -Force ; `
Expand All @@ -26,8 +32,6 @@ RUN md c:\\helixtmp && pushd c:\\helixtmp &&`
New-Item -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\' -Name 'corerun.exe' -Force -ErrorAction SilentlyContinue ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 2 -Name DumpType -Force ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 'C:\cores' -Name DumpFolder -Force ; `
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 2 -Name DumpCount -Force && `
setx /M PATH "%PATH%;C:\Python;C:\python\scripts"

WORKDIR C:\\Work
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\corerun.exe' -Value 2 -Name DumpCount -Force

WORKDIR C:\Work