From 12f787d6c6678562bd106633eab643899e67fd5d Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Wed, 13 Nov 2024 16:00:49 -0800 Subject: [PATCH 1/5] Replace "docker-compose" with "docker compose" --- .env | 2 +- cpp/examples/minimal_build/README.md | 8 +- dev/README.md | 18 +- dev/archery/README.md | 2 +- .../crossbow/tests/fixtures/azure-linux.yaml | 8 +- .../fixtures/crossbow-job-no-failure.yaml | 4 +- .../crossbow/tests/fixtures/crossbow-job.yaml | 4 +- .../archery/crossbow/tests/test_reports.py | 4 +- dev/archery/archery/docker/core.py | 2 +- docker-compose.yml | 298 +++++++++--------- .../continuous_integration/archery.rst | 12 +- .../continuous_integration/docker.rst | 40 +-- docs/source/developers/cpp/development.rst | 4 +- docs/source/developers/java/building.rst | 2 +- docs/source/developers/release.rst | 2 +- python/examples/minimal_build/README.md | 12 +- .../testthat/test-backwards-compatibility.R | 4 +- r/vignettes/developers/docker.Rmd | 4 +- r/vignettes/install.Rmd | 6 +- 19 files changed, 218 insertions(+), 218 deletions(-) diff --git a/.env b/.env index 215aa49109b4c..0b1a988875f79 100644 --- a/.env +++ b/.env @@ -97,7 +97,7 @@ VCPKG="943c5ef1c8f6b5e6ced092b242c8299caae2ff01" # 2024.04.26 Release # use pulled built images in dev/tasks/python-wheels/github.windows.yml. PYTHON_WHEEL_WINDOWS_IMAGE_REVISION=2024-08-06 -# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker-compose run --rm conan". +# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker compose run --rm conan". # See https://github.com/conan-io/conan-docker-tools#readme and # https://hub.docker.com/u/conanio for available images. CONAN_BASE=gcc10 diff --git a/cpp/examples/minimal_build/README.md b/cpp/examples/minimal_build/README.md index 9f889f6adda1a..008ecaa7b7a17 100644 --- a/cpp/examples/minimal_build/README.md +++ b/cpp/examples/minimal_build/README.md @@ -37,7 +37,7 @@ minimal Ubuntu image with a basic C++ toolchain. Just open a terminal in this directory and run the following commands: ```bash -docker-compose run --rm minimal +docker compose run --rm minimal ``` Note that this example mounts two volumes inside the Docker image: @@ -52,7 +52,7 @@ create a statically-linked executable with bundled dependencies. To run it on Linux, you can use the above Docker image: ```bash -docker-compose run --rm static +docker compose run --rm static ``` On macOS, you can use the `run_static.sh` but you must set some environment @@ -79,10 +79,10 @@ call run_static.bat You can also use static libraries of Arrow's dependencies from the system. To run this configuration, set `ARROW_DEPENDENCY_SOURCE=SYSTEM` for `run_static.sh`. You can use -`docker-compose` for this too: +`docker compose` for this too: ```bash -docker-compose run --rm static-system-dependency +docker compose run --rm static-system-dependency ``` [docker-compose]: https://docs.docker.com/compose/ diff --git a/dev/README.md b/dev/README.md index b04dd35b1c1ff..d35dd231bbc42 100644 --- a/dev/README.md +++ b/dev/README.md @@ -134,10 +134,10 @@ docker build -t arrow_integration_xenial_base -f docker_common/Dockerfile.xenial ## HDFS C++ / Python support ```shell -docker-compose build conda-cpp -docker-compose build conda-python -docker-compose build conda-python-hdfs -docker-compose run --rm conda-python-hdfs +docker compose build conda-cpp +docker compose build conda-python +docker compose build conda-python-hdfs +docker compose run --rm conda-python-hdfs ``` ## Apache Spark Integration Tests @@ -150,10 +150,10 @@ related unit tests in Spark for Java and Python. Any errors will exit with a non-zero value. To run, use the following command: ```shell -docker-compose build conda-cpp -docker-compose build conda-python -docker-compose build conda-python-spark -docker-compose run --rm conda-python-spark +docker compose build conda-cpp +docker compose build conda-python +docker compose build conda-python-spark +docker compose run --rm conda-python-spark ``` If you already are building Spark, these commands will map your local Maven @@ -162,7 +162,7 @@ Be aware, that docker write files as root, which can cause problems for maven on the host. ```shell -docker-compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark +docker compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark ``` NOTE: If the Java API has breaking changes, a patched version of Spark might diff --git a/dev/archery/README.md b/dev/archery/README.md index 0b9d5c743d122..a82e9edaa577e 100644 --- a/dev/archery/README.md +++ b/dev/archery/README.md @@ -35,7 +35,7 @@ to use the functionality of it: To install: `pip install -e "arrow/dev/archery[lint]"` * benchmark – to run Arrow benchmarks using Archery To install: `pip install -e "arrow/dev/archery[benchmark]"` -* docker – to run docker-compose based tasks more easily +* docker – to run docker compose based tasks more easily To install: `pip install -e "arrow/dev/archery[docker]"` * release – release related helpers To install: `pip install -e "arrow/dev/archery[release]"` diff --git a/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml b/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml index ff54205e347b4..d3577a3b5a935 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml @@ -31,8 +31,8 @@ jobs: - script: | set -ex docker -v - docker-compose -v + docker compose -v cd arrow - docker-compose pull --ignore-pull-failures r - docker-compose build r - displayName: Docker build \ No newline at end of file + docker compose pull --ignore-pull-failures r + docker compose build r + displayName: Docker build diff --git a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml index 68915a3f3df70..38d1882bd2fc0 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml @@ -16,8 +16,8 @@ tasks: artifacts: [] params: commands: - - docker-compose build cpp-cmake32 - - docker-compose run cpp-cmake32 + - docker compose build cpp-cmake32 + - docker compose run cpp-cmake32 branch: ursabot-1-circle-docker-cpp-cmake32 commit: a56b077c8d1b891a7935048e5672bf6fc07599ec _status: !TaskStatus diff --git a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml index e8224ef01a37f..e5944bc4352ff 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml @@ -16,8 +16,8 @@ tasks: artifacts: [] params: commands: - - docker-compose build cpp-cmake32 - - docker-compose run cpp-cmake32 + - docker compose build cpp-cmake32 + - docker compose run cpp-cmake32 branch: ursabot-1-circle-docker-cpp-cmake32 commit: a56b077c8d1b891a7935048e5672bf6fc07599ec _status: !TaskStatus diff --git a/dev/archery/archery/crossbow/tests/test_reports.py b/dev/archery/archery/crossbow/tests/test_reports.py index 601d3af1ff1c5..620b4c78bbc71 100644 --- a/dev/archery/archery/crossbow/tests/test_reports.py +++ b/dev/archery/archery/crossbow/tests/test_reports.py @@ -96,8 +96,8 @@ def test_crossbow_export_report(load_fixture): 'https://github.com/apache/crossbow/tree/' 'ursabot-1-circle-docker-cpp-cmake32', 'circle', - {'commands': ['docker-compose build cpp-cmake32', - 'docker-compose run cpp-cmake32']}, + {'commands': ['docker compose build cpp-cmake32', + 'docker compose run cpp-cmake32']}, 'docker-tests/circle.linux.yml', 'f766a1d615dd1b7ee706d05102e579195951a61c' ] diff --git a/dev/archery/archery/docker/core.py b/dev/archery/archery/docker/core.py index 2bc2a9939e4b9..fd559101b6e3e 100644 --- a/dev/archery/archery/docker/core.py +++ b/dev/archery/archery/docker/core.py @@ -130,7 +130,7 @@ def _read_config(self, config_path, compose_bin): '`x-hierarchy`'.format(name) ) - # trigger docker-compose's own validation + # trigger docker compose's own validation if self.using_docker: compose = Docker() args = ['compose'] diff --git a/docker-compose.yml b/docker-compose.yml index 0882121d598bb..a43a1e3c9a6f0 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -22,8 +22,8 @@ # defaults are set in .env file. # # Example: -# $ ARCH=arm64v8 docker-compose build ubuntu-cpp -# $ ARCH=arm64v8 docker-compose run ubuntu-cpp +# $ ARCH=arm64v8 docker compose build ubuntu-cpp +# $ ARCH=arm64v8 docker compose run ubuntu-cpp # # # Coredumps @@ -46,10 +46,10 @@ # # The setup attempts to generate coredumps by default, but the correct paths # above must be set. In order to disable the coredump generation set -# ULIMIT_CORE environment variable to 0 before running docker-compose +# ULIMIT_CORE environment variable to 0 before running docker compose # (or by setting it in .env file): # -# ULIMIT_CORE=0 docker-compose run --rm conda-cpp +# ULIMIT_CORE=0 docker compose run --rm conda-cpp # # See more in cpp/build-support/run-test.sh::print_coredumps @@ -98,11 +98,11 @@ x-hierarchy: # This section is used by the archery tool to enable building nested images, # so it is enough to call: # archery docker run debian-ruby - # instead of a sequence of docker-compose commands: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose build debian-ruby - # docker-compose run --rm debian-ruby + # instead of a sequence of docker compose commands: + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose build debian-ruby + # docker compose run --rm debian-ruby # # Each node must be either a string scalar of a list containing the # descendant images if any. Archery checks that all node has a corresponding @@ -205,19 +205,19 @@ services: ################################# C++ ####################################### # Release build: - # docker-compose run -e ARROW_BUILD_TYPE=release conda-cpp|debian-cpp|... + # docker compose run -e ARROW_BUILD_TYPE=release conda-cpp|debian-cpp|... # Shared only: - # docker-compose run -e ARROW_BUILD_STATIC=OFF conda-cpp|debian-cpp|... + # docker compose run -e ARROW_BUILD_STATIC=OFF conda-cpp|debian-cpp|... # Static only: - # docker-compose run \ + # docker compose run \ # -e ARROW_BUILD_SHARED=OFF \ # -e ARROW_TEST_LINKAGE=static \ # conda-cpp|debian-cpp|... alpine-linux-cpp: # Usage: - # docker-compose build alpine-linux-cpp - # docker-compose run --rm alpine-linux-cpp + # docker compose build alpine-linux-cpp + # docker compose run --rm alpine-linux-cpp # Parameters: # ALPINE_LINUX: 3.16 # ARCH: amd64, arm64v8, ... @@ -248,8 +248,8 @@ services: # Base image for conda builds. # # Usage: - # docker-compose build conda - # docker-compose run --rm conda + # docker compose build conda + # docker compose run --rm conda # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda @@ -267,9 +267,9 @@ services: # C++ build in conda environment, including the doxygen docs. # # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose run --rm conda-cpp + # docker compose build conda + # docker compose build conda-cpp + # docker compose run --rm conda-cpp # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda-cpp @@ -303,9 +303,9 @@ services: conda-cpp-valgrind: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose run --rm conda-cpp-valgrind + # docker compose build conda + # docker compose build conda-cpp + # docker compose run --rm conda-cpp-valgrind # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda-cpp @@ -340,8 +340,8 @@ services: debian-cpp: # Usage: - # docker-compose build debian-cpp - # docker-compose run --rm debian-cpp + # docker compose build debian-cpp + # docker compose run --rm debian-cpp # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -372,8 +372,8 @@ services: ubuntu-cpp: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose run --rm ubuntu-cpp + # docker compose build ubuntu-cpp + # docker compose run --rm ubuntu-cpp # Parameters: # ARCH: amd64, arm64v8, s390x, ... # UBUNTU: 20.04, 22.04, 24.04 @@ -408,8 +408,8 @@ services: ubuntu-cpp-static: # Usage: - # docker-compose build ubuntu-cpp-static - # docker-compose run --rm ubuntu-cpp-static + # docker compose build ubuntu-cpp-static + # docker compose run --rm ubuntu-cpp-static # Parameters: # ARCH: amd64, arm64v8, s390x, ... # UBUNTU: 20.04, 22.04 @@ -529,8 +529,8 @@ services: ubuntu-cuda-cpp: # Usage: - # docker-compose build cuda-cpp - # docker-compose run --rm cuda-cpp + # docker compose build cuda-cpp + # docker compose run --rm cuda-cpp # Parameters: # ARCH: amd64 # CUDA: @@ -577,8 +577,8 @@ services: ubuntu-cpp-sanitizer: # Usage: - # docker-compose build ubuntu-cpp-sanitizer - # docker-compose run --rm ubuntu-cpp-sanitizer + # docker compose build ubuntu-cpp-sanitizer + # docker compose run --rm ubuntu-cpp-sanitizer # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04, ... @@ -624,8 +624,8 @@ services: ubuntu-cpp-thread-sanitizer: # Usage: - # docker-compose build ubuntu-cpp-thread-sanitizer - # docker-compose run --rm ubuntu-cpp-thread-sanitizer + # docker compose build ubuntu-cpp-thread-sanitizer + # docker compose run --rm ubuntu-cpp-thread-sanitizer # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -657,8 +657,8 @@ services: ubuntu-cpp-emscripten: # Usage: - # docker-compose build ubuntu-cpp-emscripten - # docker-compose run --rm ubuntu-cpp-emscripten + # docker compose build ubuntu-cpp-emscripten + # docker compose run --rm ubuntu-cpp-emscripten # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04 @@ -682,8 +682,8 @@ services: fedora-cpp: # Usage: - # docker-compose build fedora-cpp - # docker-compose run --rm fedora-cpp + # docker compose build fedora-cpp + # docker compose run --rm fedora-cpp # Parameters: # ARCH: amd64, arm64v8, ... # FEDORA: 39 @@ -711,7 +711,7 @@ services: # Base service for Conan. # # Usage: - # docker-compose run --rm conan + # docker compose run --rm conan # Parameters: # CONAN_BASE: gcc11, gcc11-armv7, ... # CONAN_VERSION: 1.62.0 @@ -735,9 +735,9 @@ services: debian-c-glib: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose run --rm debian-c-glib + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose run --rm debian-c-glib # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -763,9 +763,9 @@ services: ubuntu-c-glib: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-c-glib - # docker-compose run --rm ubuntu-c-glib + # docker compose build ubuntu-cpp + # docker compose build ubuntu-c-glib + # docker compose run --rm ubuntu-c-glib # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -792,10 +792,10 @@ services: debian-ruby: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose build debian-ruby - # docker-compose run --rm debian-ruby + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose build debian-ruby + # docker compose run --rm debian-ruby # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -822,10 +822,10 @@ services: ubuntu-ruby: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-c-glib - # docker-compose build ubuntu-ruby - # docker-compose run --rm ubuntu-ruby + # docker compose build ubuntu-cpp + # docker compose build ubuntu-c-glib + # docker compose build ubuntu-ruby + # docker compose run --rm ubuntu-ruby # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -848,10 +848,10 @@ services: conda-python: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose run --rm conda-python + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose run --rm conda-python # Parameters: # ARCH: amd64, arm32v7 # PYTHON: 3.9, 3.10, 3.11, 3.12 @@ -878,8 +878,8 @@ services: conda-python-emscripten: # Usage: - # docker-compose build conda-python-emscripten - # docker-compose run --rm conda-python-emscripten + # docker compose build conda-python-emscripten + # docker compose run --rm conda-python-emscripten # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04 @@ -910,9 +910,9 @@ services: ubuntu-cuda-python: # Usage: - # docker-compose build cuda-cpp - # docker-compose build cuda-python - # docker-compose run --rm cuda-python + # docker compose build cuda-cpp + # docker compose build cuda-python + # docker compose run --rm cuda-python # Parameters: # ARCH: amd64 # CUDA: @@ -957,9 +957,9 @@ services: debian-python: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-python - # docker-compose run --rm debian-python + # docker compose build debian-cpp + # docker compose build debian-python + # docker compose run --rm debian-python # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -979,9 +979,9 @@ services: ubuntu-python: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python - # docker-compose run --rm ubuntu-python + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python + # docker compose run --rm ubuntu-python # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04, 24.04 @@ -1001,8 +1001,8 @@ services: ubuntu-swift: # Usage: - # docker-compose build ubuntu-swift - # docker-compose run --rm ubuntu-swift + # docker compose build ubuntu-swift + # docker compose run --rm ubuntu-swift # Parameters: image: ubuntu-swift build: @@ -1016,9 +1016,9 @@ services: fedora-python: # Usage: - # docker-compose build fedora-cpp - # docker-compose build fedora-python - # docker-compose run --rm fedora-python + # docker compose build fedora-cpp + # docker compose build fedora-python + # docker compose run --rm fedora-python # Parameters: # ARCH: amd64, arm64v8, ... # FEDORA: 39 @@ -1040,8 +1040,8 @@ services: python-sdist: # Usage: - # docker-compose build python-sdist - # docker-compose run --rm python-sdist + # docker compose build python-sdist + # docker compose run --rm python-sdist # Parameters: # PYARROW_VERSION: The pyarrow version for sdist such as "3.0.0" image: ${REPO}:python-sdist @@ -1058,9 +1058,9 @@ services: ubuntu-python-sdist-test: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python-sdist-test - # docker-compose run --rm ubuntu-python-sdist-test + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python-sdist-test + # docker compose run --rm ubuntu-python-sdist-test # Parameters: # ARCH: amd64, arm64v8, ... # PYARROW_VERSION: The test target pyarrow version such as "3.0.0" @@ -1296,14 +1296,14 @@ services: conda-python-pandas: # Possible $PANDAS parameters: # - `latest`: latest release - # - `master`: git master branch, use `docker-compose run --no-cache` + # - `master`: git master branch, use `docker compose run --no-cache` # - ``: specific version available on conda-forge # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-pandas - # docker-compose run --rm conda-python-pandas + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-pandas + # docker compose run --rm conda-python-pandas image: ${REPO}:${ARCH}-conda-python-${PYTHON}-pandas-${PANDAS} build: context: . @@ -1391,14 +1391,14 @@ services: conda-python-dask: # Possible $DASK parameters: # - `latest`: latest release - # - `master`: git master branch, use `docker-compose run --no-cache` + # - `master`: git master branch, use `docker compose run --no-cache` # - ``: specific version available on conda-forge # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-dask - # docker-compose run --rm conda-python-dask + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-dask + # docker compose run --rm conda-python-dask image: ${REPO}:${ARCH}-conda-python-${PYTHON}-dask-${DASK} build: context: . @@ -1421,11 +1421,11 @@ services: conda-python-substrait: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-substrait - # docker-compose run --rm conda-python-substrait + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-substrait + # docker compose run --rm conda-python-substrait image: ${REPO}:${ARCH}-conda-python-${PYTHON} build: context: . @@ -1448,11 +1448,11 @@ services: conda-python-jpype: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-jpype - # docker-compose run --rm conda-python-jpype + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-jpype + # docker compose run --rm conda-python-jpype image: ${REPO}:${ARCH}-conda-python-${PYTHON}-jpype build: context: . @@ -1478,11 +1478,11 @@ services: conda-python-java-integration: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-java-integration - # docker-compose run --rm conda-python-java-integration + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-java-integration + # docker compose run --rm conda-python-java-integration image: ${REPO}:${ARCH}-conda-python-${PYTHON}-java-integration build: context: . @@ -1521,11 +1521,11 @@ services: conda-python-cython2: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-cython2 - # docker-compose run --rm conda-python-cython2 + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-cython2 + # docker compose run --rm conda-python-cython2 image: ${REPO}:${ARCH}-conda-python-${PYTHON}-cython2 build: context: . @@ -1571,9 +1571,9 @@ services: ubuntu-r: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-r - # docker-compose run ubuntu-r + # docker compose build ubuntu-cpp + # docker compose build ubuntu-r + # docker compose run ubuntu-r image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-r-${R} build: context: . @@ -1622,8 +1622,8 @@ services: # (including building the C++ library) on any Docker image that contains R # # Usage: - # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r - # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r + # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r + # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r image: ${REPO}:r-${R_ORG}-${R_IMAGE}-${R_TAG} build: context: . @@ -1652,8 +1652,8 @@ services: ubuntu-r-sanitizer: # Only 20.04 and amd64 supported # Usage: - # docker-compose build ubuntu-r-sanitizer - # docker-compose run ubuntu-r-sanitizer + # docker compose build ubuntu-r-sanitizer + # docker compose run ubuntu-r-sanitizer image: ${REPO}:amd64-ubuntu-20.04-r-sanitizer cap_add: # LeakSanitizer and gdb requires ptrace(2) @@ -1706,8 +1706,8 @@ services: ubuntu-r-valgrind: # Only 20.04 and amd64 supported # Usage: - # docker-compose build ubuntu-r-valgrind - # docker-compose run ubuntu-r-valgrind + # docker compose build ubuntu-r-valgrind + # docker compose run ubuntu-r-valgrind image: ${REPO}:amd64-ubuntu-20.04-r-valgrind build: context: . @@ -1733,8 +1733,8 @@ services: r-revdepcheck: # Usage: - # docker-compose build r-revdepcheck - # docker-compose run r-revdepcheck + # docker compose build r-revdepcheck + # docker compose run r-revdepcheck image: ${REPO}:r-rstudio-r-base-4.2-focal-revdepcheck build: context: . @@ -1759,8 +1759,8 @@ services: debian-js: # Usage: - # docker-compose build debian-js - # docker-compose run debian-js + # docker compose build debian-js + # docker compose run debian-js image: ${REPO}:${ARCH}-debian-${DEBIAN}-js-${NODE} build: context: . @@ -1784,8 +1784,8 @@ services: ubuntu-csharp: # Usage: - # docker-compose build ubuntu-csharp - # docker-compose run ubuntu-csharp + # docker compose build ubuntu-csharp + # docker compose run ubuntu-csharp image: ${REPO}:${ARCH}-ubuntu-22.04-csharp-${DOTNET} build: context: . @@ -1808,8 +1808,8 @@ services: java: # Usage: - # docker-compose build java - # docker-compose run java + # docker compose build java + # docker compose run java # Parameters: # MAVEN: 3.9.5 # JDK: 11, 17, 21 @@ -1827,9 +1827,9 @@ services: conda-integration: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-integration - # docker-compose run conda-integration + # docker compose build conda-cpp + # docker compose build conda-integration + # docker compose run conda-integration image: ${REPO}:${ARCH}-conda-integration build: context: . @@ -1862,10 +1862,10 @@ services: debian-docs: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-python - # docker-compose build debian-docs - # docker-compose run --rm debian-docs + # docker compose build debian-cpp + # docker compose build debian-python + # docker compose build debian-docs + # docker compose run --rm debian-docs image: ${REPO}:${ARCH}-debian-${DEBIAN}-docs build: context: . @@ -1912,9 +1912,9 @@ services: ubuntu-lint: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-lint - # docker-compose run ubuntu-lint + # docker compose build ubuntu-cpp + # docker compose build ubuntu-lint + # docker compose run ubuntu-lint image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-lint build: context: . @@ -1972,10 +1972,10 @@ services: conda-python-hdfs: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-hdfs - # docker-compose run conda-python-hdfs + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-hdfs + # docker compose run conda-python-hdfs image: ${REPO}:${ARCH}-conda-python-${PYTHON}-hdfs-${HDFS} build: context: . @@ -2013,10 +2013,10 @@ services: conda-python-spark: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-spark - # docker-compose run conda-python-spark + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-spark + # docker compose run conda-python-spark image: ${REPO}:${ARCH}-conda-python-${PYTHON}-spark-${SPARK} build: context: . @@ -2065,8 +2065,8 @@ services: almalinux-verify-rc: # Usage: - # docker-compose build almalinux-verify-rc - # docker-compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 almalinux-verify-rc + # docker compose build almalinux-verify-rc + # docker compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 almalinux-verify-rc # Parameters: # ALMALINUX: 8 image: ${REPO}:${ARCH}-almalinux-${ALMALINUX}-verify-rc @@ -2093,8 +2093,8 @@ services: ubuntu-verify-rc: # Usage: - # docker-compose build ubuntu-verify-rc - # docker-compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 ubuntu-verify-rc + # docker compose build ubuntu-verify-rc + # docker compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 ubuntu-verify-rc # Parameters: # UBUNTU: 20.04, 22.04 image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-verify-rc diff --git a/docs/source/developers/continuous_integration/archery.rst b/docs/source/developers/continuous_integration/archery.rst index 2580693480ef1..f14140405a0d1 100644 --- a/docs/source/developers/continuous_integration/archery.rst +++ b/docs/source/developers/continuous_integration/archery.rst @@ -36,7 +36,7 @@ the top level directory install Archery by using the command $ pip install -e "dev/archery[all]" Many operations in Archery make use of `Docker `_ -and `docker-compose `_, which you may +and `docker compose `_, which you may also want to install. Usage @@ -63,7 +63,7 @@ You can inspect Archery usage by passing the ``--help`` flag: benchmark Arrow benchmarking. build Initialize an Arrow C++ build crossbow Schedule packaging tasks or nightly builds on CI services. - docker Interact with docker-compose based builds. + docker Interact with docker compose based builds. integration Execute protocol and Flight integration tests linking Quick and dirty utilities for checking library linkage. lint Check Arrow source tree for errors @@ -79,16 +79,16 @@ help output, for example: $ archery docker --help Usage: archery docker [OPTIONS] COMMAND [ARGS]... - Interact with docker-compose based builds. + Interact with docker compose based builds. Options: --src Specify Arrow source directory. --help Show this message and exit. Commands: - images List the available docker-compose images. - push Push the generated docker-compose image. - run Execute docker-compose builds. + images List the available docker compose images. + push Push the generated docker compose image. + run Execute docker compose builds. A more detailed introduction to using Docker with Archery is available at :doc:`docker`. diff --git a/docs/source/developers/continuous_integration/docker.rst b/docs/source/developers/continuous_integration/docker.rst index 129b5d0bcf135..fbe020a306f95 100644 --- a/docs/source/developers/continuous_integration/docker.rst +++ b/docs/source/developers/continuous_integration/docker.rst @@ -22,7 +22,7 @@ Running Docker Builds Most of our Linux based Continuous Integration tasks are decoupled from public CI services using `Docker `_ and -`docker-compose `_. Keeping the CI configuration +`docker compose `_. Keeping the CI configuration minimal makes local reproducibility possible. Usage @@ -46,17 +46,17 @@ Examples archery docker run conda-python -Archery calls the following docker-compose commands: +Archery calls the following docker compose commands: .. code:: bash - docker-compose pull --ignore-pull-failures conda-cpp - docker-compose pull --ignore-pull-failures conda-python - docker-compose build conda-cpp - docker-compose build conda-python - docker-compose run --rm conda-python + docker compose pull --ignore-pull-failures conda-cpp + docker compose pull --ignore-pull-failures conda-python + docker compose build conda-cpp + docker compose build conda-python + docker compose run --rm conda-python -**Show the docker-compose commands instead of executing them:** +**Show the docker compose commands instead of executing them:** .. code:: bash @@ -72,9 +72,9 @@ Which translates to: .. code:: bash - docker-compose build --no-cache conda-cpp - docker-compose build --no-cache conda-python - docker-compose run --rm conda-python + docker compose build --no-cache conda-cpp + docker compose build --no-cache conda-python + docker compose run --rm conda-python **To disable the cache only for the leaf image:** @@ -92,12 +92,12 @@ Which translates to: .. code:: bash export PANDAS=upstream_devel - docker-compose pull --ignore-pull-failures conda-cpp - docker-compose pull --ignore-pull-failures conda-python - docker-compose build conda-cpp - docker-compose build conda-python - docker-compose build --no-cache conda-python-pandas - docker-compose run --rm conda-python-pandas + docker compose pull --ignore-pull-failures conda-cpp + docker compose pull --ignore-pull-failures conda-python + docker compose build conda-cpp + docker compose build conda-python + docker compose build --no-cache conda-python-pandas + docker compose run --rm conda-python-pandas Note that it doesn't pull the conda-python-pandas image and disable the cache when building it. @@ -135,7 +135,7 @@ can be useful to skip the build phases: Most of the build scripts used within the containers can be configured through environment variables. Pass them using ``--env`` or ``-e`` CLI options - -similar to the ``docker run`` and ``docker-compose run`` interface. +similar to the ``docker run`` and ``docker compose run`` interface. .. code:: bash @@ -182,13 +182,13 @@ whole ``.docker`` directory). Development ----------- -The docker-compose configuration is tuned towards reusable development +The docker compose configuration is tuned towards reusable development containers using hierarchical images. For example multiple language bindings are dependent on the C++ implementation, so instead of redefining the C++ environment multiple Dockerfiles, we can reuse the exact same base C++ image when building Glib, Ruby, R and Python bindings. This reduces duplication and streamlines maintenance, but makes the -docker-compose configuration more complicated. +docker compose configuration more complicated. .. _docker-build-parameters: diff --git a/docs/source/developers/cpp/development.rst b/docs/source/developers/cpp/development.rst index 719d57c9387dc..882f85cc734d5 100644 --- a/docs/source/developers/cpp/development.rst +++ b/docs/source/developers/cpp/development.rst @@ -155,12 +155,12 @@ example:: -- clang-format found at /home/user/tools/bin/clang-format ...snip... -To make linting more reproducible for everyone, we provide a ``docker-compose`` +To make linting more reproducible for everyone, we provide a ``docker compose`` target that is executable from the root of the repository: .. code-block:: - $ docker-compose run ubuntu-lint + $ docker compose run ubuntu-lint Alternatively, on an open pull request, the comment bot can format C++ code for you (it will push a commit to the branch that can then be pulled). Just diff --git a/docs/source/developers/java/building.rst b/docs/source/developers/java/building.rst index 3904841de9c5a..372d44045f07f 100644 --- a/docs/source/developers/java/building.rst +++ b/docs/source/developers/java/building.rst @@ -79,7 +79,7 @@ Docker compose $ cd arrow/java $ export JAVA_HOME= $ java --version - $ docker-compose run java + $ docker compose run java Archery ~~~~~~~ diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index 55f3d5603ed05..8527eb10582ae 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -68,7 +68,7 @@ generated properly. - Install Python 3 as python - Create dev/release/.env from dev/release/.env.example. See the comments in dev/release/.env.example how to set each variable. - Setup :ref:`Crossbow` as defined. - - Have Docker and docker-compose installed. + - Have Docker and docker compose installed. Before creating a Release Candidate diff --git a/python/examples/minimal_build/README.md b/python/examples/minimal_build/README.md index d5cda25ce2b72..e9d595b603103 100644 --- a/python/examples/minimal_build/README.md +++ b/python/examples/minimal_build/README.md @@ -42,13 +42,13 @@ docker run --rm -t -i -v $PWD:/io -v $PWD/../../..:/arrow arrow_fedora_minimal / Alternatively you can use [Docker Compose][docker-compose] to build and run using: ``` -docker-compose build +docker compose build # With conda -docker-compose run --rm minimal-fedora-conda +docker compose run --rm minimal-fedora-conda # With pip -docker-compose run --rm minimal-fedora-venv +docker compose run --rm minimal-fedora-venv ``` ## Ubuntu 20.04 @@ -69,13 +69,13 @@ docker run --rm -t -i -v $PWD:/io -v $PWD/../../..:/arrow arrow_ubuntu_minimal / Alternatively you can use [Docker Compose][docker-compose] to build and run using: ``` -docker-compose build +docker compose build # With conda -docker-compose run --rm minimal-ubuntu-conda +docker compose run --rm minimal-ubuntu-conda # With pip -docker-compose run --rm minimal-ubuntu-venv +docker compose run --rm minimal-ubuntu-venv ``` ## Building on Fedora - Podman and SELinux diff --git a/r/tests/testthat/test-backwards-compatibility.R b/r/tests/testthat/test-backwards-compatibility.R index 5f804b02dcee7..81ef4bb86d2d5 100644 --- a/r/tests/testthat/test-backwards-compatibility.R +++ b/r/tests/testthat/test-backwards-compatibility.R @@ -23,8 +23,8 @@ # to setup a linux distribution and use RStudio's public package manager binary # repo to install the old version. The following commands should be run at the # root of the arrow repo directory and might need slight adjustments. -# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker-compose build --no-cache r -# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker-compose run r /bin/bash +# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker compose build --no-cache r +# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker compose run r /bin/bash # R # options(repos = "https://packagemanager.rstudio.com/all/__linux__/focal/latest") # remotes::install_version("arrow", version = "1.0.1") diff --git a/r/vignettes/developers/docker.Rmd b/r/vignettes/developers/docker.Rmd index 13f60904c9484..57432e594cd1b 100644 --- a/r/vignettes/developers/docker.Rmd +++ b/r/vignettes/developers/docker.Rmd @@ -87,8 +87,8 @@ If you wanted to run [RHub's latest `ubuntu-release` image](https://hub.docker.c run: ```shell -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r ``` ### Example - Using Archery diff --git a/r/vignettes/install.Rmd b/r/vignettes/install.Rmd index c7b8251ccc99b..07d858c6cd62a 100644 --- a/r/vignettes/install.Rmd +++ b/r/vignettes/install.Rmd @@ -500,12 +500,12 @@ it should be as minimal as possible, containing only R and the dependencies it requires. For reference, see the images that [R-hub](https://github.com/r-hub/rhub-linux-builders) uses. -You can test the arrow R package installation using the `docker-compose` +You can test the arrow R package installation using the `docker compose` setup included in the `apache/arrow` git repository. For example, ``` -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r ``` installs the arrow R package, including libarrow, on the From b38aee669b328c906a04089a41ea9b4fa01fbc81 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Wed, 13 Nov 2024 16:16:35 -0800 Subject: [PATCH 2/5] More replacements --- dev/archery/archery/docker/cli.py | 30 ++++++++++++------------ dev/archery/archery/docker/core.py | 8 +++---- dev/tasks/python-wheels/github.linux.yml | 2 +- docker-compose.yml | 26 ++++++++++---------- docs/source/format/Integration.rst | 4 ++-- 5 files changed, 35 insertions(+), 35 deletions(-) diff --git a/dev/archery/archery/docker/cli.py b/dev/archery/archery/docker/cli.py index 3375d63306a10..eeb01716faa71 100644 --- a/dev/archery/archery/docker/cli.py +++ b/dev/archery/archery/docker/cli.py @@ -67,7 +67,7 @@ def _execute(self, *args, **kwargs): def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, using_docker_buildx): """ - Interact with docker-compose based builds. + Interact with docker compose based builds. """ ctx.ensure_object(dict) @@ -78,8 +78,8 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, "try to pass the arrow source directory explicitly.".format(src) ) - # take the docker-compose parameters like PYTHON, PANDAS, UBUNTU from the - # environment variables to keep the usage similar to docker-compose + # take the docker compose parameters like PYTHON, PANDAS, UBUNTU from the + # environment variables to keep the usage similar to docker compose using_docker_cli |= using_docker_buildx compose_bin = ("docker-compose" if using_legacy_docker_compose else "docker compose") @@ -98,7 +98,7 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, @click.pass_obj def check_config(obj): """ - Validate docker-compose configuration. + Validate docker compose configuration. """ # executes the body of the docker function above which does the validation # during the configuration loading @@ -113,7 +113,7 @@ def check_config(obj): @click.pass_obj def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures): """ - Execute docker-compose pull. + Execute docker compose pull. """ compose = obj['compose'] @@ -143,7 +143,7 @@ def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures): @click.pass_obj def docker_build(obj, image, *, force_pull, use_cache, use_leaf_cache): """ - Execute docker-compose builds. + Execute docker compose builds. """ compose = obj['compose'] @@ -195,7 +195,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, build_only, use_cache, use_leaf_cache, resource_limit, volume): """ - Execute docker-compose builds. + Execute docker compose builds. To see the available builds run `archery docker images`. @@ -207,7 +207,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, # execute the builds but disable the image pulling archery docker run --no-cache conda-python - # pass a docker-compose parameter, like the python version + # pass a docker compose parameter, like the python version PYTHON=3.12 archery docker run conda-python # disable the cache only for the leaf image @@ -265,7 +265,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, help='Docker repository password') @click.pass_obj def docker_compose_push(obj, image, user, password): - """Push the generated docker-compose image.""" + """Push the generated docker compose image.""" compose = obj['compose'] compose.push(image, user=user, password=password) @@ -273,7 +273,7 @@ def docker_compose_push(obj, image, user, password): @docker.command('images') @click.pass_obj def docker_compose_images(obj): - """List the available docker-compose images.""" + """List the available docker compose images.""" compose = obj['compose'] click.echo('Available images:') for image in compose.images(): @@ -283,14 +283,14 @@ def docker_compose_images(obj): @docker.command('info') @click.argument('service_name') @click.option('--show', '-s', required=False, - help="Show only specific docker-compose key. Examples of keys:" + help="Show only specific docker compose key. Examples of keys:" " command, environment, build, dockerfile") @click.pass_obj def docker_compose_info(obj, service_name, show): - """Show docker-compose definition info for service_name. + """Show docker compose definition info for service_name. - SERVICE_NAME is the name of the docker service defined on - the docker-compose. Look at `archery docker images` output for names. + SERVICE_NAME is the name of the docker service defined in + docker-compose.yml. Look at `archery docker images` output for names. """ compose = obj['compose'] try: @@ -299,6 +299,6 @@ def docker_compose_info(obj, service_name, show): click.echo(f'Service name {service_name} could not be found', err=True) sys.exit(1) else: - click.echo(f'Service {service_name} docker-compose config:') + click.echo(f'Service {service_name} docker compose config:') output = "\n".join(compose.info(service, show)) click.echo(output) diff --git a/dev/archery/archery/docker/core.py b/dev/archery/archery/docker/core.py index fd559101b6e3e..66c4a48c1e73c 100644 --- a/dev/archery/archery/docker/core.py +++ b/dev/archery/archery/docker/core.py @@ -142,7 +142,7 @@ def _read_config(self, config_path, compose_bin): stderr=subprocess.PIPE, stdout=subprocess.PIPE) if result.returncode != 0: - # strip the intro line of docker-compose errors + # strip the intro line of docker compose errors errors += result.stderr.decode().splitlines() if errors: @@ -203,7 +203,7 @@ def formatdict(d, template): ) msg = ( "`{cmd}` exited with a non-zero exit code {code}, see the " - "process log above.\n\nThe docker-compose command was " + "process log above.\n\nThe {compose_bin} command was " "invoked with the following parameters:\n\nDefaults defined " "in .env:\n{dotenv}\n\nArchery was called with:\n{params}" ) @@ -406,7 +406,7 @@ def run(self, service_name, command=None, *, env=None, volumes=None, cmd = service.get('command', '') if cmd: # service command might be already defined as a list - # on the docker-compose yaml file. + # in docker-compose.yml. if isinstance(cmd, list): cmd = shlex.join(cmd) # Match behaviour from docker compose @@ -418,7 +418,7 @@ def run(self, service_name, command=None, *, env=None, volumes=None, # execute as a plain docker cli command self._execute_docker('run', '--rm', *args) else: - # execute as a docker-compose command + # execute as a docker compose command args.append(service_name) if command is not None: args.append(command) diff --git a/dev/tasks/python-wheels/github.linux.yml b/dev/tasks/python-wheels/github.linux.yml index 7a1c8fb4f9d80..a822c4d338e49 100644 --- a/dev/tasks/python-wheels/github.linux.yml +++ b/dev/tasks/python-wheels/github.linux.yml @@ -66,7 +66,7 @@ jobs: archery docker run python-wheel-manylinux-test-imports archery docker run python-wheel-manylinux-test-unittests - # Free-threaded wheels need to be tested using a different docker-compose service + # Free-threaded wheels need to be tested using a different docker compose service - name: Test free-threaded wheel if: | '{{ python_abi_tag }}' == 'cp313t' diff --git a/docker-compose.yml b/docker-compose.yml index a43a1e3c9a6f0..5714d524a960e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1090,9 +1090,9 @@ services: ubuntu-python-313-freethreading: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python-313-freethreading - # docker-compose run --rm ubuntu-python-313-freethreading + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python-313-freethreading + # docker compose run --rm ubuntu-python-313-freethreading # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04, 24.04 @@ -1328,11 +1328,11 @@ services: conda-python-no-numpy: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-no-numpy - # docker-compose run --rm conda-python-no-numpy + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-no-numpy + # docker compose run --rm conda-python-no-numpy image: ${REPO}:${ARCH}-conda-python-${PYTHON}-no-numpy build: context: . @@ -1545,11 +1545,11 @@ services: conda-python-cpython-debug: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-cpython-debug - # docker-compose run --rm conda-python-cpython-debug + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-cpython-debug + # docker compose run --rm conda-python-cpython-debug image: ${REPO}:${ARCH}-conda-python-${PYTHON}-cpython-debug build: context: . diff --git a/docs/source/format/Integration.rst b/docs/source/format/Integration.rst index f76aa0fefcf27..e31fcd5bf1636 100644 --- a/docs/source/format/Integration.rst +++ b/docs/source/format/Integration.rst @@ -144,12 +144,12 @@ To run all tests, including Flight and C Data Interface integration tests, do: archery integration --with-all --run-flight --run-ipc --run-c-data Note that we run these tests in continuous integration, and the CI job uses -docker-compose. You may also run the docker-compose job locally, or at least +docker compose. You may also run the docker compose job locally, or at least refer to it if you have questions about how to build other languages or enable certain tests. See :ref:`docker-builds` for more information about the project's -``docker-compose`` configuration. +``docker compose`` configuration. .. _format_json_integration: From ff78655a2f43a70ff881de6276fe52404aa8570b Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Wed, 13 Nov 2024 16:38:27 -0800 Subject: [PATCH 3/5] Fix broken fix --- dev/archery/archery/docker/core.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/dev/archery/archery/docker/core.py b/dev/archery/archery/docker/core.py index 66c4a48c1e73c..db7fd1cbd5ce2 100644 --- a/dev/archery/archery/docker/core.py +++ b/dev/archery/archery/docker/core.py @@ -203,7 +203,7 @@ def formatdict(d, template): ) msg = ( "`{cmd}` exited with a non-zero exit code {code}, see the " - "process log above.\n\nThe {compose_bin} command was " + "process log above.\n\nThe {bin} command was " "invoked with the following parameters:\n\nDefaults defined " "in .env:\n{dotenv}\n\nArchery was called with:\n{params}" ) @@ -211,6 +211,7 @@ def formatdict(d, template): msg.format( cmd=' '.join(e.cmd), code=e.returncode, + bin=self.bin, dotenv=formatdict(self.config.dotenv, template=' {}: {}'), params=formatdict( self.config.params, template=' export {}={}' From d653d558b660dec6bb07f45d66fae867cb3f0bc6 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 14 Nov 2024 11:28:13 -0800 Subject: [PATCH 4/5] Update docs/source/developers/continuous_integration/archery.rst Co-authored-by: Sutou Kouhei --- docs/source/developers/continuous_integration/archery.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/developers/continuous_integration/archery.rst b/docs/source/developers/continuous_integration/archery.rst index f14140405a0d1..68c02c3a65be5 100644 --- a/docs/source/developers/continuous_integration/archery.rst +++ b/docs/source/developers/continuous_integration/archery.rst @@ -36,7 +36,7 @@ the top level directory install Archery by using the command $ pip install -e "dev/archery[all]" Many operations in Archery make use of `Docker `_ -and `docker compose `_, which you may +and `Docker Compose `_, which you may also want to install. Usage From 71a365a52c5b921aa2828d440e3b4500b6a1d861 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 14 Nov 2024 11:38:20 -0800 Subject: [PATCH 5/5] Use "Docker Compose" where appropriate --- dev/archery/archery/docker/cli.py | 20 +++++++++---------- dev/archery/archery/docker/core.py | 4 ++-- dev/tasks/python-wheels/github.linux.yml | 2 +- .../continuous_integration/archery.rst | 10 +++++----- .../continuous_integration/docker.rst | 10 +++++----- docs/source/developers/release.rst | 2 +- docs/source/format/Integration.rst | 2 +- 7 files changed, 25 insertions(+), 25 deletions(-) diff --git a/dev/archery/archery/docker/cli.py b/dev/archery/archery/docker/cli.py index eeb01716faa71..5f9cea872a2a8 100644 --- a/dev/archery/archery/docker/cli.py +++ b/dev/archery/archery/docker/cli.py @@ -67,7 +67,7 @@ def _execute(self, *args, **kwargs): def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, using_docker_buildx): """ - Interact with docker compose based builds. + Interact with Docker Compose based builds. """ ctx.ensure_object(dict) @@ -78,7 +78,7 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, "try to pass the arrow source directory explicitly.".format(src) ) - # take the docker compose parameters like PYTHON, PANDAS, UBUNTU from the + # take the Docker Compose parameters like PYTHON, PANDAS, UBUNTU from the # environment variables to keep the usage similar to docker compose using_docker_cli |= using_docker_buildx compose_bin = ("docker-compose" if using_legacy_docker_compose @@ -98,7 +98,7 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, @click.pass_obj def check_config(obj): """ - Validate docker compose configuration. + Validate Docker Compose configuration. """ # executes the body of the docker function above which does the validation # during the configuration loading @@ -143,7 +143,7 @@ def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures): @click.pass_obj def docker_build(obj, image, *, force_pull, use_cache, use_leaf_cache): """ - Execute docker compose builds. + Execute Docker Compose builds. """ compose = obj['compose'] @@ -195,7 +195,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, build_only, use_cache, use_leaf_cache, resource_limit, volume): """ - Execute docker compose builds. + Execute Docker Compose builds. To see the available builds run `archery docker images`. @@ -207,7 +207,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, # execute the builds but disable the image pulling archery docker run --no-cache conda-python - # pass a docker compose parameter, like the python version + # pass a Docker Compose parameter, like the python version PYTHON=3.12 archery docker run conda-python # disable the cache only for the leaf image @@ -265,7 +265,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, help='Docker repository password') @click.pass_obj def docker_compose_push(obj, image, user, password): - """Push the generated docker compose image.""" + """Push the generated Docker Compose image.""" compose = obj['compose'] compose.push(image, user=user, password=password) @@ -273,7 +273,7 @@ def docker_compose_push(obj, image, user, password): @docker.command('images') @click.pass_obj def docker_compose_images(obj): - """List the available docker compose images.""" + """List the available Docker Compose images.""" compose = obj['compose'] click.echo('Available images:') for image in compose.images(): @@ -287,7 +287,7 @@ def docker_compose_images(obj): " command, environment, build, dockerfile") @click.pass_obj def docker_compose_info(obj, service_name, show): - """Show docker compose definition info for service_name. + """Show Docker Compose definition info for service_name. SERVICE_NAME is the name of the docker service defined in docker-compose.yml. Look at `archery docker images` output for names. @@ -299,6 +299,6 @@ def docker_compose_info(obj, service_name, show): click.echo(f'Service name {service_name} could not be found', err=True) sys.exit(1) else: - click.echo(f'Service {service_name} docker compose config:') + click.echo(f'Service {service_name} Docker Compose config:') output = "\n".join(compose.info(service, show)) click.echo(output) diff --git a/dev/archery/archery/docker/core.py b/dev/archery/archery/docker/core.py index db7fd1cbd5ce2..55269835972ba 100644 --- a/dev/archery/archery/docker/core.py +++ b/dev/archery/archery/docker/core.py @@ -130,7 +130,7 @@ def _read_config(self, config_path, compose_bin): '`x-hierarchy`'.format(name) ) - # trigger docker compose's own validation + # trigger Docker Compose's own validation if self.using_docker: compose = Docker() args = ['compose'] @@ -410,7 +410,7 @@ def run(self, service_name, command=None, *, env=None, volumes=None, # in docker-compose.yml. if isinstance(cmd, list): cmd = shlex.join(cmd) - # Match behaviour from docker compose + # Match behaviour from Docker Compose # to interpolate environment variables # https://docs.docker.com/compose/compose-file/12-interpolation/ cmd = cmd.replace("$$", "$") diff --git a/dev/tasks/python-wheels/github.linux.yml b/dev/tasks/python-wheels/github.linux.yml index a822c4d338e49..363e78ab51ee0 100644 --- a/dev/tasks/python-wheels/github.linux.yml +++ b/dev/tasks/python-wheels/github.linux.yml @@ -66,7 +66,7 @@ jobs: archery docker run python-wheel-manylinux-test-imports archery docker run python-wheel-manylinux-test-unittests - # Free-threaded wheels need to be tested using a different docker compose service + # Free-threaded wheels need to be tested using a different Docker Compose service - name: Test free-threaded wheel if: | '{{ python_abi_tag }}' == 'cp313t' diff --git a/docs/source/developers/continuous_integration/archery.rst b/docs/source/developers/continuous_integration/archery.rst index 68c02c3a65be5..ce82fc728c943 100644 --- a/docs/source/developers/continuous_integration/archery.rst +++ b/docs/source/developers/continuous_integration/archery.rst @@ -63,7 +63,7 @@ You can inspect Archery usage by passing the ``--help`` flag: benchmark Arrow benchmarking. build Initialize an Arrow C++ build crossbow Schedule packaging tasks or nightly builds on CI services. - docker Interact with docker compose based builds. + docker Interact with Docker Compose based builds. integration Execute protocol and Flight integration tests linking Quick and dirty utilities for checking library linkage. lint Check Arrow source tree for errors @@ -79,16 +79,16 @@ help output, for example: $ archery docker --help Usage: archery docker [OPTIONS] COMMAND [ARGS]... - Interact with docker compose based builds. + Interact with Docker Compose based builds. Options: --src Specify Arrow source directory. --help Show this message and exit. Commands: - images List the available docker compose images. - push Push the generated docker compose image. - run Execute docker compose builds. + images List the available Docker Compose images. + push Push the generated Docker Compose image. + run Execute Docker Compose builds. A more detailed introduction to using Docker with Archery is available at :doc:`docker`. diff --git a/docs/source/developers/continuous_integration/docker.rst b/docs/source/developers/continuous_integration/docker.rst index fbe020a306f95..1cd811904a1fe 100644 --- a/docs/source/developers/continuous_integration/docker.rst +++ b/docs/source/developers/continuous_integration/docker.rst @@ -22,13 +22,13 @@ Running Docker Builds Most of our Linux based Continuous Integration tasks are decoupled from public CI services using `Docker `_ and -`docker compose `_. Keeping the CI configuration +`Docker Compose `_. Keeping the CI configuration minimal makes local reproducibility possible. Usage ----- -There are multiple ways to execute the docker based builds. +There are multiple ways to execute the Docker based builds. The recommended way is to use the :ref:`Archery ` tool: Examples @@ -56,7 +56,7 @@ Archery calls the following docker compose commands: docker compose build conda-python docker compose run --rm conda-python -**Show the docker compose commands instead of executing them:** +**Show the Docker Compose commands instead of executing them:** .. code:: bash @@ -182,13 +182,13 @@ whole ``.docker`` directory). Development ----------- -The docker compose configuration is tuned towards reusable development +The Docker Compose configuration is tuned towards reusable development containers using hierarchical images. For example multiple language bindings are dependent on the C++ implementation, so instead of redefining the C++ environment multiple Dockerfiles, we can reuse the exact same base C++ image when building Glib, Ruby, R and Python bindings. This reduces duplication and streamlines maintenance, but makes the -docker compose configuration more complicated. +Docker Compose configuration more complicated. .. _docker-build-parameters: diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index 8527eb10582ae..53550752d497b 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -68,7 +68,7 @@ generated properly. - Install Python 3 as python - Create dev/release/.env from dev/release/.env.example. See the comments in dev/release/.env.example how to set each variable. - Setup :ref:`Crossbow` as defined. - - Have Docker and docker compose installed. + - Have Docker and Docker Compose installed. Before creating a Release Candidate diff --git a/docs/source/format/Integration.rst b/docs/source/format/Integration.rst index e31fcd5bf1636..1d99773bfa64c 100644 --- a/docs/source/format/Integration.rst +++ b/docs/source/format/Integration.rst @@ -144,7 +144,7 @@ To run all tests, including Flight and C Data Interface integration tests, do: archery integration --with-all --run-flight --run-ipc --run-c-data Note that we run these tests in continuous integration, and the CI job uses -docker compose. You may also run the docker compose job locally, or at least +Docker Compose. You may also run the Docker Compose job locally, or at least refer to it if you have questions about how to build other languages or enable certain tests.