diff --git a/.env b/.env index 215aa49109b4c..0b1a988875f79 100644 --- a/.env +++ b/.env @@ -97,7 +97,7 @@ VCPKG="943c5ef1c8f6b5e6ced092b242c8299caae2ff01" # 2024.04.26 Release # use pulled built images in dev/tasks/python-wheels/github.windows.yml. PYTHON_WHEEL_WINDOWS_IMAGE_REVISION=2024-08-06 -# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker-compose run --rm conan". +# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker compose run --rm conan". # See https://github.com/conan-io/conan-docker-tools#readme and # https://hub.docker.com/u/conanio for available images. CONAN_BASE=gcc10 diff --git a/cpp/examples/minimal_build/README.md b/cpp/examples/minimal_build/README.md index 9f889f6adda1a..008ecaa7b7a17 100644 --- a/cpp/examples/minimal_build/README.md +++ b/cpp/examples/minimal_build/README.md @@ -37,7 +37,7 @@ minimal Ubuntu image with a basic C++ toolchain. Just open a terminal in this directory and run the following commands: ```bash -docker-compose run --rm minimal +docker compose run --rm minimal ``` Note that this example mounts two volumes inside the Docker image: @@ -52,7 +52,7 @@ create a statically-linked executable with bundled dependencies. To run it on Linux, you can use the above Docker image: ```bash -docker-compose run --rm static +docker compose run --rm static ``` On macOS, you can use the `run_static.sh` but you must set some environment @@ -79,10 +79,10 @@ call run_static.bat You can also use static libraries of Arrow's dependencies from the system. To run this configuration, set `ARROW_DEPENDENCY_SOURCE=SYSTEM` for `run_static.sh`. You can use -`docker-compose` for this too: +`docker compose` for this too: ```bash -docker-compose run --rm static-system-dependency +docker compose run --rm static-system-dependency ``` [docker-compose]: https://docs.docker.com/compose/ diff --git a/dev/README.md b/dev/README.md index b04dd35b1c1ff..d35dd231bbc42 100644 --- a/dev/README.md +++ b/dev/README.md @@ -134,10 +134,10 @@ docker build -t arrow_integration_xenial_base -f docker_common/Dockerfile.xenial ## HDFS C++ / Python support ```shell -docker-compose build conda-cpp -docker-compose build conda-python -docker-compose build conda-python-hdfs -docker-compose run --rm conda-python-hdfs +docker compose build conda-cpp +docker compose build conda-python +docker compose build conda-python-hdfs +docker compose run --rm conda-python-hdfs ``` ## Apache Spark Integration Tests @@ -150,10 +150,10 @@ related unit tests in Spark for Java and Python. Any errors will exit with a non-zero value. To run, use the following command: ```shell -docker-compose build conda-cpp -docker-compose build conda-python -docker-compose build conda-python-spark -docker-compose run --rm conda-python-spark +docker compose build conda-cpp +docker compose build conda-python +docker compose build conda-python-spark +docker compose run --rm conda-python-spark ``` If you already are building Spark, these commands will map your local Maven @@ -162,7 +162,7 @@ Be aware, that docker write files as root, which can cause problems for maven on the host. ```shell -docker-compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark +docker compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark ``` NOTE: If the Java API has breaking changes, a patched version of Spark might diff --git a/dev/archery/README.md b/dev/archery/README.md index 0b9d5c743d122..a82e9edaa577e 100644 --- a/dev/archery/README.md +++ b/dev/archery/README.md @@ -35,7 +35,7 @@ to use the functionality of it: To install: `pip install -e "arrow/dev/archery[lint]"` * benchmark – to run Arrow benchmarks using Archery To install: `pip install -e "arrow/dev/archery[benchmark]"` -* docker – to run docker-compose based tasks more easily +* docker – to run docker compose based tasks more easily To install: `pip install -e "arrow/dev/archery[docker]"` * release – release related helpers To install: `pip install -e "arrow/dev/archery[release]"` diff --git a/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml b/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml index ff54205e347b4..d3577a3b5a935 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml @@ -31,8 +31,8 @@ jobs: - script: | set -ex docker -v - docker-compose -v + docker compose -v cd arrow - docker-compose pull --ignore-pull-failures r - docker-compose build r - displayName: Docker build \ No newline at end of file + docker compose pull --ignore-pull-failures r + docker compose build r + displayName: Docker build diff --git a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml index 68915a3f3df70..38d1882bd2fc0 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job-no-failure.yaml @@ -16,8 +16,8 @@ tasks: artifacts: [] params: commands: - - docker-compose build cpp-cmake32 - - docker-compose run cpp-cmake32 + - docker compose build cpp-cmake32 + - docker compose run cpp-cmake32 branch: ursabot-1-circle-docker-cpp-cmake32 commit: a56b077c8d1b891a7935048e5672bf6fc07599ec _status: !TaskStatus diff --git a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml index e8224ef01a37f..e5944bc4352ff 100644 --- a/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml +++ b/dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml @@ -16,8 +16,8 @@ tasks: artifacts: [] params: commands: - - docker-compose build cpp-cmake32 - - docker-compose run cpp-cmake32 + - docker compose build cpp-cmake32 + - docker compose run cpp-cmake32 branch: ursabot-1-circle-docker-cpp-cmake32 commit: a56b077c8d1b891a7935048e5672bf6fc07599ec _status: !TaskStatus diff --git a/dev/archery/archery/crossbow/tests/test_reports.py b/dev/archery/archery/crossbow/tests/test_reports.py index 601d3af1ff1c5..620b4c78bbc71 100644 --- a/dev/archery/archery/crossbow/tests/test_reports.py +++ b/dev/archery/archery/crossbow/tests/test_reports.py @@ -96,8 +96,8 @@ def test_crossbow_export_report(load_fixture): 'https://github.com/apache/crossbow/tree/' 'ursabot-1-circle-docker-cpp-cmake32', 'circle', - {'commands': ['docker-compose build cpp-cmake32', - 'docker-compose run cpp-cmake32']}, + {'commands': ['docker compose build cpp-cmake32', + 'docker compose run cpp-cmake32']}, 'docker-tests/circle.linux.yml', 'f766a1d615dd1b7ee706d05102e579195951a61c' ] diff --git a/dev/archery/archery/docker/cli.py b/dev/archery/archery/docker/cli.py index 3375d63306a10..5f9cea872a2a8 100644 --- a/dev/archery/archery/docker/cli.py +++ b/dev/archery/archery/docker/cli.py @@ -67,7 +67,7 @@ def _execute(self, *args, **kwargs): def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, using_docker_buildx): """ - Interact with docker-compose based builds. + Interact with Docker Compose based builds. """ ctx.ensure_object(dict) @@ -78,8 +78,8 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, "try to pass the arrow source directory explicitly.".format(src) ) - # take the docker-compose parameters like PYTHON, PANDAS, UBUNTU from the - # environment variables to keep the usage similar to docker-compose + # take the Docker Compose parameters like PYTHON, PANDAS, UBUNTU from the + # environment variables to keep the usage similar to docker compose using_docker_cli |= using_docker_buildx compose_bin = ("docker-compose" if using_legacy_docker_compose else "docker compose") @@ -98,7 +98,7 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli, @click.pass_obj def check_config(obj): """ - Validate docker-compose configuration. + Validate Docker Compose configuration. """ # executes the body of the docker function above which does the validation # during the configuration loading @@ -113,7 +113,7 @@ def check_config(obj): @click.pass_obj def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures): """ - Execute docker-compose pull. + Execute docker compose pull. """ compose = obj['compose'] @@ -143,7 +143,7 @@ def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures): @click.pass_obj def docker_build(obj, image, *, force_pull, use_cache, use_leaf_cache): """ - Execute docker-compose builds. + Execute Docker Compose builds. """ compose = obj['compose'] @@ -195,7 +195,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, build_only, use_cache, use_leaf_cache, resource_limit, volume): """ - Execute docker-compose builds. + Execute Docker Compose builds. To see the available builds run `archery docker images`. @@ -207,7 +207,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, # execute the builds but disable the image pulling archery docker run --no-cache conda-python - # pass a docker-compose parameter, like the python version + # pass a Docker Compose parameter, like the python version PYTHON=3.12 archery docker run conda-python # disable the cache only for the leaf image @@ -265,7 +265,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build, help='Docker repository password') @click.pass_obj def docker_compose_push(obj, image, user, password): - """Push the generated docker-compose image.""" + """Push the generated Docker Compose image.""" compose = obj['compose'] compose.push(image, user=user, password=password) @@ -273,7 +273,7 @@ def docker_compose_push(obj, image, user, password): @docker.command('images') @click.pass_obj def docker_compose_images(obj): - """List the available docker-compose images.""" + """List the available Docker Compose images.""" compose = obj['compose'] click.echo('Available images:') for image in compose.images(): @@ -283,14 +283,14 @@ def docker_compose_images(obj): @docker.command('info') @click.argument('service_name') @click.option('--show', '-s', required=False, - help="Show only specific docker-compose key. Examples of keys:" + help="Show only specific docker compose key. Examples of keys:" " command, environment, build, dockerfile") @click.pass_obj def docker_compose_info(obj, service_name, show): - """Show docker-compose definition info for service_name. + """Show Docker Compose definition info for service_name. - SERVICE_NAME is the name of the docker service defined on - the docker-compose. Look at `archery docker images` output for names. + SERVICE_NAME is the name of the docker service defined in + docker-compose.yml. Look at `archery docker images` output for names. """ compose = obj['compose'] try: @@ -299,6 +299,6 @@ def docker_compose_info(obj, service_name, show): click.echo(f'Service name {service_name} could not be found', err=True) sys.exit(1) else: - click.echo(f'Service {service_name} docker-compose config:') + click.echo(f'Service {service_name} Docker Compose config:') output = "\n".join(compose.info(service, show)) click.echo(output) diff --git a/dev/archery/archery/docker/core.py b/dev/archery/archery/docker/core.py index 2bc2a9939e4b9..55269835972ba 100644 --- a/dev/archery/archery/docker/core.py +++ b/dev/archery/archery/docker/core.py @@ -130,7 +130,7 @@ def _read_config(self, config_path, compose_bin): '`x-hierarchy`'.format(name) ) - # trigger docker-compose's own validation + # trigger Docker Compose's own validation if self.using_docker: compose = Docker() args = ['compose'] @@ -142,7 +142,7 @@ def _read_config(self, config_path, compose_bin): stderr=subprocess.PIPE, stdout=subprocess.PIPE) if result.returncode != 0: - # strip the intro line of docker-compose errors + # strip the intro line of docker compose errors errors += result.stderr.decode().splitlines() if errors: @@ -203,7 +203,7 @@ def formatdict(d, template): ) msg = ( "`{cmd}` exited with a non-zero exit code {code}, see the " - "process log above.\n\nThe docker-compose command was " + "process log above.\n\nThe {bin} command was " "invoked with the following parameters:\n\nDefaults defined " "in .env:\n{dotenv}\n\nArchery was called with:\n{params}" ) @@ -211,6 +211,7 @@ def formatdict(d, template): msg.format( cmd=' '.join(e.cmd), code=e.returncode, + bin=self.bin, dotenv=formatdict(self.config.dotenv, template=' {}: {}'), params=formatdict( self.config.params, template=' export {}={}' @@ -406,10 +407,10 @@ def run(self, service_name, command=None, *, env=None, volumes=None, cmd = service.get('command', '') if cmd: # service command might be already defined as a list - # on the docker-compose yaml file. + # in docker-compose.yml. if isinstance(cmd, list): cmd = shlex.join(cmd) - # Match behaviour from docker compose + # Match behaviour from Docker Compose # to interpolate environment variables # https://docs.docker.com/compose/compose-file/12-interpolation/ cmd = cmd.replace("$$", "$") @@ -418,7 +419,7 @@ def run(self, service_name, command=None, *, env=None, volumes=None, # execute as a plain docker cli command self._execute_docker('run', '--rm', *args) else: - # execute as a docker-compose command + # execute as a docker compose command args.append(service_name) if command is not None: args.append(command) diff --git a/dev/tasks/python-wheels/github.linux.yml b/dev/tasks/python-wheels/github.linux.yml index 7a1c8fb4f9d80..363e78ab51ee0 100644 --- a/dev/tasks/python-wheels/github.linux.yml +++ b/dev/tasks/python-wheels/github.linux.yml @@ -66,7 +66,7 @@ jobs: archery docker run python-wheel-manylinux-test-imports archery docker run python-wheel-manylinux-test-unittests - # Free-threaded wheels need to be tested using a different docker-compose service + # Free-threaded wheels need to be tested using a different Docker Compose service - name: Test free-threaded wheel if: | '{{ python_abi_tag }}' == 'cp313t' diff --git a/docker-compose.yml b/docker-compose.yml index 0882121d598bb..5714d524a960e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -22,8 +22,8 @@ # defaults are set in .env file. # # Example: -# $ ARCH=arm64v8 docker-compose build ubuntu-cpp -# $ ARCH=arm64v8 docker-compose run ubuntu-cpp +# $ ARCH=arm64v8 docker compose build ubuntu-cpp +# $ ARCH=arm64v8 docker compose run ubuntu-cpp # # # Coredumps @@ -46,10 +46,10 @@ # # The setup attempts to generate coredumps by default, but the correct paths # above must be set. In order to disable the coredump generation set -# ULIMIT_CORE environment variable to 0 before running docker-compose +# ULIMIT_CORE environment variable to 0 before running docker compose # (or by setting it in .env file): # -# ULIMIT_CORE=0 docker-compose run --rm conda-cpp +# ULIMIT_CORE=0 docker compose run --rm conda-cpp # # See more in cpp/build-support/run-test.sh::print_coredumps @@ -98,11 +98,11 @@ x-hierarchy: # This section is used by the archery tool to enable building nested images, # so it is enough to call: # archery docker run debian-ruby - # instead of a sequence of docker-compose commands: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose build debian-ruby - # docker-compose run --rm debian-ruby + # instead of a sequence of docker compose commands: + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose build debian-ruby + # docker compose run --rm debian-ruby # # Each node must be either a string scalar of a list containing the # descendant images if any. Archery checks that all node has a corresponding @@ -205,19 +205,19 @@ services: ################################# C++ ####################################### # Release build: - # docker-compose run -e ARROW_BUILD_TYPE=release conda-cpp|debian-cpp|... + # docker compose run -e ARROW_BUILD_TYPE=release conda-cpp|debian-cpp|... # Shared only: - # docker-compose run -e ARROW_BUILD_STATIC=OFF conda-cpp|debian-cpp|... + # docker compose run -e ARROW_BUILD_STATIC=OFF conda-cpp|debian-cpp|... # Static only: - # docker-compose run \ + # docker compose run \ # -e ARROW_BUILD_SHARED=OFF \ # -e ARROW_TEST_LINKAGE=static \ # conda-cpp|debian-cpp|... alpine-linux-cpp: # Usage: - # docker-compose build alpine-linux-cpp - # docker-compose run --rm alpine-linux-cpp + # docker compose build alpine-linux-cpp + # docker compose run --rm alpine-linux-cpp # Parameters: # ALPINE_LINUX: 3.16 # ARCH: amd64, arm64v8, ... @@ -248,8 +248,8 @@ services: # Base image for conda builds. # # Usage: - # docker-compose build conda - # docker-compose run --rm conda + # docker compose build conda + # docker compose run --rm conda # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda @@ -267,9 +267,9 @@ services: # C++ build in conda environment, including the doxygen docs. # # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose run --rm conda-cpp + # docker compose build conda + # docker compose build conda-cpp + # docker compose run --rm conda-cpp # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda-cpp @@ -303,9 +303,9 @@ services: conda-cpp-valgrind: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose run --rm conda-cpp-valgrind + # docker compose build conda + # docker compose build conda-cpp + # docker compose run --rm conda-cpp-valgrind # Parameters: # ARCH: amd64, arm32v7 image: ${REPO}:${ARCH}-conda-cpp @@ -340,8 +340,8 @@ services: debian-cpp: # Usage: - # docker-compose build debian-cpp - # docker-compose run --rm debian-cpp + # docker compose build debian-cpp + # docker compose run --rm debian-cpp # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -372,8 +372,8 @@ services: ubuntu-cpp: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose run --rm ubuntu-cpp + # docker compose build ubuntu-cpp + # docker compose run --rm ubuntu-cpp # Parameters: # ARCH: amd64, arm64v8, s390x, ... # UBUNTU: 20.04, 22.04, 24.04 @@ -408,8 +408,8 @@ services: ubuntu-cpp-static: # Usage: - # docker-compose build ubuntu-cpp-static - # docker-compose run --rm ubuntu-cpp-static + # docker compose build ubuntu-cpp-static + # docker compose run --rm ubuntu-cpp-static # Parameters: # ARCH: amd64, arm64v8, s390x, ... # UBUNTU: 20.04, 22.04 @@ -529,8 +529,8 @@ services: ubuntu-cuda-cpp: # Usage: - # docker-compose build cuda-cpp - # docker-compose run --rm cuda-cpp + # docker compose build cuda-cpp + # docker compose run --rm cuda-cpp # Parameters: # ARCH: amd64 # CUDA: @@ -577,8 +577,8 @@ services: ubuntu-cpp-sanitizer: # Usage: - # docker-compose build ubuntu-cpp-sanitizer - # docker-compose run --rm ubuntu-cpp-sanitizer + # docker compose build ubuntu-cpp-sanitizer + # docker compose run --rm ubuntu-cpp-sanitizer # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04, ... @@ -624,8 +624,8 @@ services: ubuntu-cpp-thread-sanitizer: # Usage: - # docker-compose build ubuntu-cpp-thread-sanitizer - # docker-compose run --rm ubuntu-cpp-thread-sanitizer + # docker compose build ubuntu-cpp-thread-sanitizer + # docker compose run --rm ubuntu-cpp-thread-sanitizer # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -657,8 +657,8 @@ services: ubuntu-cpp-emscripten: # Usage: - # docker-compose build ubuntu-cpp-emscripten - # docker-compose run --rm ubuntu-cpp-emscripten + # docker compose build ubuntu-cpp-emscripten + # docker compose run --rm ubuntu-cpp-emscripten # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04 @@ -682,8 +682,8 @@ services: fedora-cpp: # Usage: - # docker-compose build fedora-cpp - # docker-compose run --rm fedora-cpp + # docker compose build fedora-cpp + # docker compose run --rm fedora-cpp # Parameters: # ARCH: amd64, arm64v8, ... # FEDORA: 39 @@ -711,7 +711,7 @@ services: # Base service for Conan. # # Usage: - # docker-compose run --rm conan + # docker compose run --rm conan # Parameters: # CONAN_BASE: gcc11, gcc11-armv7, ... # CONAN_VERSION: 1.62.0 @@ -735,9 +735,9 @@ services: debian-c-glib: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose run --rm debian-c-glib + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose run --rm debian-c-glib # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -763,9 +763,9 @@ services: ubuntu-c-glib: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-c-glib - # docker-compose run --rm ubuntu-c-glib + # docker compose build ubuntu-cpp + # docker compose build ubuntu-c-glib + # docker compose run --rm ubuntu-c-glib # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -792,10 +792,10 @@ services: debian-ruby: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-c-glib - # docker-compose build debian-ruby - # docker-compose run --rm debian-ruby + # docker compose build debian-cpp + # docker compose build debian-c-glib + # docker compose build debian-ruby + # docker compose run --rm debian-ruby # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -822,10 +822,10 @@ services: ubuntu-ruby: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-c-glib - # docker-compose build ubuntu-ruby - # docker-compose run --rm ubuntu-ruby + # docker compose build ubuntu-cpp + # docker compose build ubuntu-c-glib + # docker compose build ubuntu-ruby + # docker compose run --rm ubuntu-ruby # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04 @@ -848,10 +848,10 @@ services: conda-python: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose run --rm conda-python + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose run --rm conda-python # Parameters: # ARCH: amd64, arm32v7 # PYTHON: 3.9, 3.10, 3.11, 3.12 @@ -878,8 +878,8 @@ services: conda-python-emscripten: # Usage: - # docker-compose build conda-python-emscripten - # docker-compose run --rm conda-python-emscripten + # docker compose build conda-python-emscripten + # docker compose run --rm conda-python-emscripten # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04 @@ -910,9 +910,9 @@ services: ubuntu-cuda-python: # Usage: - # docker-compose build cuda-cpp - # docker-compose build cuda-python - # docker-compose run --rm cuda-python + # docker compose build cuda-cpp + # docker compose build cuda-python + # docker compose run --rm cuda-python # Parameters: # ARCH: amd64 # CUDA: @@ -957,9 +957,9 @@ services: debian-python: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-python - # docker-compose run --rm debian-python + # docker compose build debian-cpp + # docker compose build debian-python + # docker compose run --rm debian-python # Parameters: # ARCH: amd64, arm64v8, ... # DEBIAN: 12 @@ -979,9 +979,9 @@ services: ubuntu-python: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python - # docker-compose run --rm ubuntu-python + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python + # docker compose run --rm ubuntu-python # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 22.04, 24.04 @@ -1001,8 +1001,8 @@ services: ubuntu-swift: # Usage: - # docker-compose build ubuntu-swift - # docker-compose run --rm ubuntu-swift + # docker compose build ubuntu-swift + # docker compose run --rm ubuntu-swift # Parameters: image: ubuntu-swift build: @@ -1016,9 +1016,9 @@ services: fedora-python: # Usage: - # docker-compose build fedora-cpp - # docker-compose build fedora-python - # docker-compose run --rm fedora-python + # docker compose build fedora-cpp + # docker compose build fedora-python + # docker compose run --rm fedora-python # Parameters: # ARCH: amd64, arm64v8, ... # FEDORA: 39 @@ -1040,8 +1040,8 @@ services: python-sdist: # Usage: - # docker-compose build python-sdist - # docker-compose run --rm python-sdist + # docker compose build python-sdist + # docker compose run --rm python-sdist # Parameters: # PYARROW_VERSION: The pyarrow version for sdist such as "3.0.0" image: ${REPO}:python-sdist @@ -1058,9 +1058,9 @@ services: ubuntu-python-sdist-test: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python-sdist-test - # docker-compose run --rm ubuntu-python-sdist-test + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python-sdist-test + # docker compose run --rm ubuntu-python-sdist-test # Parameters: # ARCH: amd64, arm64v8, ... # PYARROW_VERSION: The test target pyarrow version such as "3.0.0" @@ -1090,9 +1090,9 @@ services: ubuntu-python-313-freethreading: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-python-313-freethreading - # docker-compose run --rm ubuntu-python-313-freethreading + # docker compose build ubuntu-cpp + # docker compose build ubuntu-python-313-freethreading + # docker compose run --rm ubuntu-python-313-freethreading # Parameters: # ARCH: amd64, arm64v8, ... # UBUNTU: 20.04, 22.04, 24.04 @@ -1296,14 +1296,14 @@ services: conda-python-pandas: # Possible $PANDAS parameters: # - `latest`: latest release - # - `master`: git master branch, use `docker-compose run --no-cache` + # - `master`: git master branch, use `docker compose run --no-cache` # - ``: specific version available on conda-forge # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-pandas - # docker-compose run --rm conda-python-pandas + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-pandas + # docker compose run --rm conda-python-pandas image: ${REPO}:${ARCH}-conda-python-${PYTHON}-pandas-${PANDAS} build: context: . @@ -1328,11 +1328,11 @@ services: conda-python-no-numpy: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-no-numpy - # docker-compose run --rm conda-python-no-numpy + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-no-numpy + # docker compose run --rm conda-python-no-numpy image: ${REPO}:${ARCH}-conda-python-${PYTHON}-no-numpy build: context: . @@ -1391,14 +1391,14 @@ services: conda-python-dask: # Possible $DASK parameters: # - `latest`: latest release - # - `master`: git master branch, use `docker-compose run --no-cache` + # - `master`: git master branch, use `docker compose run --no-cache` # - ``: specific version available on conda-forge # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-dask - # docker-compose run --rm conda-python-dask + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-dask + # docker compose run --rm conda-python-dask image: ${REPO}:${ARCH}-conda-python-${PYTHON}-dask-${DASK} build: context: . @@ -1421,11 +1421,11 @@ services: conda-python-substrait: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-substrait - # docker-compose run --rm conda-python-substrait + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-substrait + # docker compose run --rm conda-python-substrait image: ${REPO}:${ARCH}-conda-python-${PYTHON} build: context: . @@ -1448,11 +1448,11 @@ services: conda-python-jpype: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-jpype - # docker-compose run --rm conda-python-jpype + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-jpype + # docker compose run --rm conda-python-jpype image: ${REPO}:${ARCH}-conda-python-${PYTHON}-jpype build: context: . @@ -1478,11 +1478,11 @@ services: conda-python-java-integration: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-java-integration - # docker-compose run --rm conda-python-java-integration + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-java-integration + # docker compose run --rm conda-python-java-integration image: ${REPO}:${ARCH}-conda-python-${PYTHON}-java-integration build: context: . @@ -1521,11 +1521,11 @@ services: conda-python-cython2: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-cython2 - # docker-compose run --rm conda-python-cython2 + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-cython2 + # docker compose run --rm conda-python-cython2 image: ${REPO}:${ARCH}-conda-python-${PYTHON}-cython2 build: context: . @@ -1545,11 +1545,11 @@ services: conda-python-cpython-debug: # Usage: - # docker-compose build conda - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-cpython-debug - # docker-compose run --rm conda-python-cpython-debug + # docker compose build conda + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-cpython-debug + # docker compose run --rm conda-python-cpython-debug image: ${REPO}:${ARCH}-conda-python-${PYTHON}-cpython-debug build: context: . @@ -1571,9 +1571,9 @@ services: ubuntu-r: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-r - # docker-compose run ubuntu-r + # docker compose build ubuntu-cpp + # docker compose build ubuntu-r + # docker compose run ubuntu-r image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-r-${R} build: context: . @@ -1622,8 +1622,8 @@ services: # (including building the C++ library) on any Docker image that contains R # # Usage: - # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r - # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r + # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r + # R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r image: ${REPO}:r-${R_ORG}-${R_IMAGE}-${R_TAG} build: context: . @@ -1652,8 +1652,8 @@ services: ubuntu-r-sanitizer: # Only 20.04 and amd64 supported # Usage: - # docker-compose build ubuntu-r-sanitizer - # docker-compose run ubuntu-r-sanitizer + # docker compose build ubuntu-r-sanitizer + # docker compose run ubuntu-r-sanitizer image: ${REPO}:amd64-ubuntu-20.04-r-sanitizer cap_add: # LeakSanitizer and gdb requires ptrace(2) @@ -1706,8 +1706,8 @@ services: ubuntu-r-valgrind: # Only 20.04 and amd64 supported # Usage: - # docker-compose build ubuntu-r-valgrind - # docker-compose run ubuntu-r-valgrind + # docker compose build ubuntu-r-valgrind + # docker compose run ubuntu-r-valgrind image: ${REPO}:amd64-ubuntu-20.04-r-valgrind build: context: . @@ -1733,8 +1733,8 @@ services: r-revdepcheck: # Usage: - # docker-compose build r-revdepcheck - # docker-compose run r-revdepcheck + # docker compose build r-revdepcheck + # docker compose run r-revdepcheck image: ${REPO}:r-rstudio-r-base-4.2-focal-revdepcheck build: context: . @@ -1759,8 +1759,8 @@ services: debian-js: # Usage: - # docker-compose build debian-js - # docker-compose run debian-js + # docker compose build debian-js + # docker compose run debian-js image: ${REPO}:${ARCH}-debian-${DEBIAN}-js-${NODE} build: context: . @@ -1784,8 +1784,8 @@ services: ubuntu-csharp: # Usage: - # docker-compose build ubuntu-csharp - # docker-compose run ubuntu-csharp + # docker compose build ubuntu-csharp + # docker compose run ubuntu-csharp image: ${REPO}:${ARCH}-ubuntu-22.04-csharp-${DOTNET} build: context: . @@ -1808,8 +1808,8 @@ services: java: # Usage: - # docker-compose build java - # docker-compose run java + # docker compose build java + # docker compose run java # Parameters: # MAVEN: 3.9.5 # JDK: 11, 17, 21 @@ -1827,9 +1827,9 @@ services: conda-integration: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-integration - # docker-compose run conda-integration + # docker compose build conda-cpp + # docker compose build conda-integration + # docker compose run conda-integration image: ${REPO}:${ARCH}-conda-integration build: context: . @@ -1862,10 +1862,10 @@ services: debian-docs: # Usage: - # docker-compose build debian-cpp - # docker-compose build debian-python - # docker-compose build debian-docs - # docker-compose run --rm debian-docs + # docker compose build debian-cpp + # docker compose build debian-python + # docker compose build debian-docs + # docker compose run --rm debian-docs image: ${REPO}:${ARCH}-debian-${DEBIAN}-docs build: context: . @@ -1912,9 +1912,9 @@ services: ubuntu-lint: # Usage: - # docker-compose build ubuntu-cpp - # docker-compose build ubuntu-lint - # docker-compose run ubuntu-lint + # docker compose build ubuntu-cpp + # docker compose build ubuntu-lint + # docker compose run ubuntu-lint image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-lint build: context: . @@ -1972,10 +1972,10 @@ services: conda-python-hdfs: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-hdfs - # docker-compose run conda-python-hdfs + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-hdfs + # docker compose run conda-python-hdfs image: ${REPO}:${ARCH}-conda-python-${PYTHON}-hdfs-${HDFS} build: context: . @@ -2013,10 +2013,10 @@ services: conda-python-spark: # Usage: - # docker-compose build conda-cpp - # docker-compose build conda-python - # docker-compose build conda-python-spark - # docker-compose run conda-python-spark + # docker compose build conda-cpp + # docker compose build conda-python + # docker compose build conda-python-spark + # docker compose run conda-python-spark image: ${REPO}:${ARCH}-conda-python-${PYTHON}-spark-${SPARK} build: context: . @@ -2065,8 +2065,8 @@ services: almalinux-verify-rc: # Usage: - # docker-compose build almalinux-verify-rc - # docker-compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 almalinux-verify-rc + # docker compose build almalinux-verify-rc + # docker compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 almalinux-verify-rc # Parameters: # ALMALINUX: 8 image: ${REPO}:${ARCH}-almalinux-${ALMALINUX}-verify-rc @@ -2093,8 +2093,8 @@ services: ubuntu-verify-rc: # Usage: - # docker-compose build ubuntu-verify-rc - # docker-compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 ubuntu-verify-rc + # docker compose build ubuntu-verify-rc + # docker compose run -e VERIFY_VERSION=6.0.1 -e VERIFY_RC=1 ubuntu-verify-rc # Parameters: # UBUNTU: 20.04, 22.04 image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-verify-rc diff --git a/docs/source/developers/continuous_integration/archery.rst b/docs/source/developers/continuous_integration/archery.rst index 2580693480ef1..ce82fc728c943 100644 --- a/docs/source/developers/continuous_integration/archery.rst +++ b/docs/source/developers/continuous_integration/archery.rst @@ -36,7 +36,7 @@ the top level directory install Archery by using the command $ pip install -e "dev/archery[all]" Many operations in Archery make use of `Docker `_ -and `docker-compose `_, which you may +and `Docker Compose `_, which you may also want to install. Usage @@ -63,7 +63,7 @@ You can inspect Archery usage by passing the ``--help`` flag: benchmark Arrow benchmarking. build Initialize an Arrow C++ build crossbow Schedule packaging tasks or nightly builds on CI services. - docker Interact with docker-compose based builds. + docker Interact with Docker Compose based builds. integration Execute protocol and Flight integration tests linking Quick and dirty utilities for checking library linkage. lint Check Arrow source tree for errors @@ -79,16 +79,16 @@ help output, for example: $ archery docker --help Usage: archery docker [OPTIONS] COMMAND [ARGS]... - Interact with docker-compose based builds. + Interact with Docker Compose based builds. Options: --src Specify Arrow source directory. --help Show this message and exit. Commands: - images List the available docker-compose images. - push Push the generated docker-compose image. - run Execute docker-compose builds. + images List the available Docker Compose images. + push Push the generated Docker Compose image. + run Execute Docker Compose builds. A more detailed introduction to using Docker with Archery is available at :doc:`docker`. diff --git a/docs/source/developers/continuous_integration/docker.rst b/docs/source/developers/continuous_integration/docker.rst index 129b5d0bcf135..1cd811904a1fe 100644 --- a/docs/source/developers/continuous_integration/docker.rst +++ b/docs/source/developers/continuous_integration/docker.rst @@ -22,13 +22,13 @@ Running Docker Builds Most of our Linux based Continuous Integration tasks are decoupled from public CI services using `Docker `_ and -`docker-compose `_. Keeping the CI configuration +`Docker Compose `_. Keeping the CI configuration minimal makes local reproducibility possible. Usage ----- -There are multiple ways to execute the docker based builds. +There are multiple ways to execute the Docker based builds. The recommended way is to use the :ref:`Archery ` tool: Examples @@ -46,17 +46,17 @@ Examples archery docker run conda-python -Archery calls the following docker-compose commands: +Archery calls the following docker compose commands: .. code:: bash - docker-compose pull --ignore-pull-failures conda-cpp - docker-compose pull --ignore-pull-failures conda-python - docker-compose build conda-cpp - docker-compose build conda-python - docker-compose run --rm conda-python + docker compose pull --ignore-pull-failures conda-cpp + docker compose pull --ignore-pull-failures conda-python + docker compose build conda-cpp + docker compose build conda-python + docker compose run --rm conda-python -**Show the docker-compose commands instead of executing them:** +**Show the Docker Compose commands instead of executing them:** .. code:: bash @@ -72,9 +72,9 @@ Which translates to: .. code:: bash - docker-compose build --no-cache conda-cpp - docker-compose build --no-cache conda-python - docker-compose run --rm conda-python + docker compose build --no-cache conda-cpp + docker compose build --no-cache conda-python + docker compose run --rm conda-python **To disable the cache only for the leaf image:** @@ -92,12 +92,12 @@ Which translates to: .. code:: bash export PANDAS=upstream_devel - docker-compose pull --ignore-pull-failures conda-cpp - docker-compose pull --ignore-pull-failures conda-python - docker-compose build conda-cpp - docker-compose build conda-python - docker-compose build --no-cache conda-python-pandas - docker-compose run --rm conda-python-pandas + docker compose pull --ignore-pull-failures conda-cpp + docker compose pull --ignore-pull-failures conda-python + docker compose build conda-cpp + docker compose build conda-python + docker compose build --no-cache conda-python-pandas + docker compose run --rm conda-python-pandas Note that it doesn't pull the conda-python-pandas image and disable the cache when building it. @@ -135,7 +135,7 @@ can be useful to skip the build phases: Most of the build scripts used within the containers can be configured through environment variables. Pass them using ``--env`` or ``-e`` CLI options - -similar to the ``docker run`` and ``docker-compose run`` interface. +similar to the ``docker run`` and ``docker compose run`` interface. .. code:: bash @@ -182,13 +182,13 @@ whole ``.docker`` directory). Development ----------- -The docker-compose configuration is tuned towards reusable development +The Docker Compose configuration is tuned towards reusable development containers using hierarchical images. For example multiple language bindings are dependent on the C++ implementation, so instead of redefining the C++ environment multiple Dockerfiles, we can reuse the exact same base C++ image when building Glib, Ruby, R and Python bindings. This reduces duplication and streamlines maintenance, but makes the -docker-compose configuration more complicated. +Docker Compose configuration more complicated. .. _docker-build-parameters: diff --git a/docs/source/developers/cpp/development.rst b/docs/source/developers/cpp/development.rst index 719d57c9387dc..882f85cc734d5 100644 --- a/docs/source/developers/cpp/development.rst +++ b/docs/source/developers/cpp/development.rst @@ -155,12 +155,12 @@ example:: -- clang-format found at /home/user/tools/bin/clang-format ...snip... -To make linting more reproducible for everyone, we provide a ``docker-compose`` +To make linting more reproducible for everyone, we provide a ``docker compose`` target that is executable from the root of the repository: .. code-block:: - $ docker-compose run ubuntu-lint + $ docker compose run ubuntu-lint Alternatively, on an open pull request, the comment bot can format C++ code for you (it will push a commit to the branch that can then be pulled). Just diff --git a/docs/source/developers/java/building.rst b/docs/source/developers/java/building.rst index 3904841de9c5a..372d44045f07f 100644 --- a/docs/source/developers/java/building.rst +++ b/docs/source/developers/java/building.rst @@ -79,7 +79,7 @@ Docker compose $ cd arrow/java $ export JAVA_HOME= $ java --version - $ docker-compose run java + $ docker compose run java Archery ~~~~~~~ diff --git a/docs/source/developers/release.rst b/docs/source/developers/release.rst index 55f3d5603ed05..53550752d497b 100644 --- a/docs/source/developers/release.rst +++ b/docs/source/developers/release.rst @@ -68,7 +68,7 @@ generated properly. - Install Python 3 as python - Create dev/release/.env from dev/release/.env.example. See the comments in dev/release/.env.example how to set each variable. - Setup :ref:`Crossbow` as defined. - - Have Docker and docker-compose installed. + - Have Docker and Docker Compose installed. Before creating a Release Candidate diff --git a/docs/source/format/Integration.rst b/docs/source/format/Integration.rst index f76aa0fefcf27..1d99773bfa64c 100644 --- a/docs/source/format/Integration.rst +++ b/docs/source/format/Integration.rst @@ -144,12 +144,12 @@ To run all tests, including Flight and C Data Interface integration tests, do: archery integration --with-all --run-flight --run-ipc --run-c-data Note that we run these tests in continuous integration, and the CI job uses -docker-compose. You may also run the docker-compose job locally, or at least +Docker Compose. You may also run the Docker Compose job locally, or at least refer to it if you have questions about how to build other languages or enable certain tests. See :ref:`docker-builds` for more information about the project's -``docker-compose`` configuration. +``docker compose`` configuration. .. _format_json_integration: diff --git a/python/examples/minimal_build/README.md b/python/examples/minimal_build/README.md index d5cda25ce2b72..e9d595b603103 100644 --- a/python/examples/minimal_build/README.md +++ b/python/examples/minimal_build/README.md @@ -42,13 +42,13 @@ docker run --rm -t -i -v $PWD:/io -v $PWD/../../..:/arrow arrow_fedora_minimal / Alternatively you can use [Docker Compose][docker-compose] to build and run using: ``` -docker-compose build +docker compose build # With conda -docker-compose run --rm minimal-fedora-conda +docker compose run --rm minimal-fedora-conda # With pip -docker-compose run --rm minimal-fedora-venv +docker compose run --rm minimal-fedora-venv ``` ## Ubuntu 20.04 @@ -69,13 +69,13 @@ docker run --rm -t -i -v $PWD:/io -v $PWD/../../..:/arrow arrow_ubuntu_minimal / Alternatively you can use [Docker Compose][docker-compose] to build and run using: ``` -docker-compose build +docker compose build # With conda -docker-compose run --rm minimal-ubuntu-conda +docker compose run --rm minimal-ubuntu-conda # With pip -docker-compose run --rm minimal-ubuntu-venv +docker compose run --rm minimal-ubuntu-venv ``` ## Building on Fedora - Podman and SELinux diff --git a/r/tests/testthat/test-backwards-compatibility.R b/r/tests/testthat/test-backwards-compatibility.R index 5f804b02dcee7..81ef4bb86d2d5 100644 --- a/r/tests/testthat/test-backwards-compatibility.R +++ b/r/tests/testthat/test-backwards-compatibility.R @@ -23,8 +23,8 @@ # to setup a linux distribution and use RStudio's public package manager binary # repo to install the old version. The following commands should be run at the # root of the arrow repo directory and might need slight adjustments. -# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker-compose build --no-cache r -# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker-compose run r /bin/bash +# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker compose build --no-cache r +# R_ORG=rstudio R_IMAGE=r-base R_TAG=4.0-focal docker compose run r /bin/bash # R # options(repos = "https://packagemanager.rstudio.com/all/__linux__/focal/latest") # remotes::install_version("arrow", version = "1.0.1") diff --git a/r/vignettes/developers/docker.Rmd b/r/vignettes/developers/docker.Rmd index 13f60904c9484..57432e594cd1b 100644 --- a/r/vignettes/developers/docker.Rmd +++ b/r/vignettes/developers/docker.Rmd @@ -87,8 +87,8 @@ If you wanted to run [RHub's latest `ubuntu-release` image](https://hub.docker.c run: ```shell -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r ``` ### Example - Using Archery diff --git a/r/vignettes/install.Rmd b/r/vignettes/install.Rmd index c7b8251ccc99b..07d858c6cd62a 100644 --- a/r/vignettes/install.Rmd +++ b/r/vignettes/install.Rmd @@ -500,12 +500,12 @@ it should be as minimal as possible, containing only R and the dependencies it requires. For reference, see the images that [R-hub](https://github.com/r-hub/rhub-linux-builders) uses. -You can test the arrow R package installation using the `docker-compose` +You can test the arrow R package installation using the `docker compose` setup included in the `apache/arrow` git repository. For example, ``` -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose build r -R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker-compose run r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose build r +R_ORG=rhub R_IMAGE=ubuntu-release R_TAG=latest docker compose run r ``` installs the arrow R package, including libarrow, on the