Skip to content

Commit 3c91348

Browse files
committed
pythonbuild: make download_to_path() concurrently safe
I'm about to enable support for parallel CPython builds. This function was currently not safe if called in parallel. This was causing setuptools / pip downloads to race writing to the same file and failing builds.
1 parent 0ed9d38 commit 3c91348

File tree

1 file changed

+11
-1
lines changed

1 file changed

+11
-1
lines changed

pythonbuild/utils.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@
1212
import os
1313
import pathlib
1414
import platform
15+
import random
1516
import stat
17+
import string
1618
import subprocess
1719
import sys
1820
import tarfile
@@ -269,7 +271,15 @@ def download_to_path(url: str, path: pathlib.Path, size: int, sha256: str):
269271

270272
path.unlink()
271273

272-
tmp = path.with_name("%s.tmp" % path.name)
274+
# Need to write to random path to avoid race conditions. If there is a
275+
# race, worst case we'll download the same file N>1 times. Meh.
276+
tmp = path.with_name(
277+
"%s.tmp%s"
278+
% (
279+
path.name,
280+
"".join(random.choices(string.ascii_uppercase + string.digits, k=8)),
281+
)
282+
)
273283

274284
for attempt in range(5):
275285
try:

0 commit comments

Comments
 (0)