Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch PT 2.4 Training SM DLC #4457

Merged
merged 7 commits into from
Dec 11, 2024
Merged

Patch PT 2.4 Training SM DLC #4457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
838960a
Patch PT 2.4 Training SM DLC
sirutBuasai Dec 10, 2024
dc04db0
pin blis
sirutBuasai Dec 10, 2024
e92d2be
build cpu pip install blis
sirutBuasai Dec 10, 2024
4d69986
build test all
sirutBuasai Dec 10, 2024
e4d59d7
build all
sirutBuasai Dec 10, 2024
44b1eba
revert dockerfile and toml
sirutBuasai Dec 11, 2024
36519ff
Merge branch 'master' into 2.4-patch
sirutBuasai Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions pytorch/training/docker/2.4/py3/Dockerfile.cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,13 +306,16 @@ RUN /opt/conda/bin/mamba install -y -c conda-forge \
shap \
# pinned for sagemaker==2.235.1
"cloudpickle==2.2.1" \
&& /opt/conda/bin/mamba clean -afy

RUN pip install --no-cache-dir -U \
# pin numpy requirement for sagemaker dependency
# requires explicit declaration of spacy, thic, blis
# pip install due to pip check conflict with conda package
spacy \
thinc \
blis \
"numpy<2" \
&& /opt/conda/bin/mamba clean -afy
"numpy<2"

# Copy workaround script for incorrect hostname
COPY changehostname.c /
Expand Down
4 changes: 2 additions & 2 deletions pytorch/training/docker/2.4/py3/Dockerfile.sagemaker.cpu.core_packages.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
{
"accelerate": {
"version_specifier": "==1.1.1",
"version_specifier": "==1.2.0",
"skip": "True"
},
"fastai": {
"version_specifier": "==2.7.18",
"skip": "True"
},
"s3torchconnector": {
"version_specifier": "==1.2.7",
"version_specifier": "==1.3.0",
"skip": "True"
},
"torchaudio": {
Expand Down
7 changes: 5 additions & 2 deletions pytorch/training/docker/2.4/py3/cu124/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,13 +488,16 @@ RUN /opt/conda/bin/mamba install -y -c conda-forge \
seaborn \
# pinned for sagemaker==2.235.1
"cloudpickle==2.2.1" \
&& /opt/conda/bin/mamba clean -afy

RUN pip install --no-cache-dir -U \
# pin numpy requirement for sagemaker dependency
# requires explicit declaration of spacy, thic, blis
# pip install due to pip check conflict with conda package
spacy \
thinc \
blis \
"numpy<2" \
&& /opt/conda/bin/mamba clean -afy
"numpy<2"

# Add SageMaker DataParallel to LD_LIBRARY_PATH
ENV LD_LIBRARY_PATH="/opt/conda/lib/python${PYTHON_SHORT_VERSION}/site-packages/smdistributed/dataparallel/lib:$LD_LIBRARY_PATH"
Expand Down
4 changes: 2 additions & 2 deletions pytorch/training/docker/2.4/py3/cu124/Dockerfile.sagemaker.gpu.core_packages.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"accelerate": {
"version_specifier": "==1.1.1",
"version_specifier": "==1.2.0",
"skip": "True"
},
"fastai": {
Expand All @@ -12,7 +12,7 @@
"skip": "True"
},
"s3torchconnector": {
"version_specifier": "==1.2.7",
"version_specifier": "==1.3.0",
"skip": "True"
},
"torchaudio": {
Expand Down
Loading