Skip to content

Commit

Permalink
GH-43570: [CI][Dev][Docs] Update references to "docker-compose" with …
Browse files Browse the repository at this point in the history
…"docker compose" (#43575)

### Rationale for this change

This is a follow-on to #43566 which was merged earlier today. See #43570 for more detail on the entire set of changes.

### What changes are included in this PR?

- Replace instances where we call or document calling `docker-compose` with `docker compose`.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

This PR includes the require user-facing changes to document things.
* GitHub Issue: #43570

Lead-authored-by: Bryce Mecum <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
  • Loading branch information
amoeba and kou authored Nov 15, 2024
1 parent 3f25672 commit 736d706
Show file tree
Hide file tree
Showing 22 changed files with 256 additions and 255 deletions.
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ VCPKG="943c5ef1c8f6b5e6ced092b242c8299caae2ff01" # 2024.04.26 Release
# use pulled built images in dev/tasks/python-wheels/github.windows.yml.
PYTHON_WHEEL_WINDOWS_IMAGE_REVISION=2024-08-06

# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker-compose run --rm conan".
# Use conanio/${CONAN_BASE}:{CONAN_VERSION} for "docker compose run --rm conan".
# See https://github.com/conan-io/conan-docker-tools#readme and
# https://hub.docker.com/u/conanio for available images.
CONAN_BASE=gcc10
Expand Down
8 changes: 4 additions & 4 deletions cpp/examples/minimal_build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ minimal Ubuntu image with a basic C++ toolchain.
Just open a terminal in this directory and run the following commands:

```bash
docker-compose run --rm minimal
docker compose run --rm minimal
```

Note that this example mounts two volumes inside the Docker image:
Expand All @@ -52,7 +52,7 @@ create a statically-linked executable with bundled dependencies.
To run it on Linux, you can use the above Docker image:

```bash
docker-compose run --rm static
docker compose run --rm static
```

On macOS, you can use the `run_static.sh` but you must set some environment
Expand All @@ -79,10 +79,10 @@ call run_static.bat
You can also use static libraries of Arrow's dependencies from the
system. To run this configuration, set
`ARROW_DEPENDENCY_SOURCE=SYSTEM` for `run_static.sh`. You can use
`docker-compose` for this too:
`docker compose` for this too:

```bash
docker-compose run --rm static-system-dependency
docker compose run --rm static-system-dependency
```

[docker-compose]: https://docs.docker.com/compose/
18 changes: 9 additions & 9 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,10 +134,10 @@ docker build -t arrow_integration_xenial_base -f docker_common/Dockerfile.xenial
## HDFS C++ / Python support

```shell
docker-compose build conda-cpp
docker-compose build conda-python
docker-compose build conda-python-hdfs
docker-compose run --rm conda-python-hdfs
docker compose build conda-cpp
docker compose build conda-python
docker compose build conda-python-hdfs
docker compose run --rm conda-python-hdfs
```

## Apache Spark Integration Tests
Expand All @@ -150,10 +150,10 @@ related unit tests in Spark for Java and Python. Any errors will exit with a
non-zero value. To run, use the following command:

```shell
docker-compose build conda-cpp
docker-compose build conda-python
docker-compose build conda-python-spark
docker-compose run --rm conda-python-spark
docker compose build conda-cpp
docker compose build conda-python
docker compose build conda-python-spark
docker compose run --rm conda-python-spark
```

If you already are building Spark, these commands will map your local Maven
Expand All @@ -162,7 +162,7 @@ Be aware, that docker write files as root, which can cause problems for maven
on the host.

```shell
docker-compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark
docker compose run --rm -v $HOME/.m2:/root/.m2 conda-python-spark
```

NOTE: If the Java API has breaking changes, a patched version of Spark might
Expand Down
2 changes: 1 addition & 1 deletion dev/archery/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ to use the functionality of it:
To install: `pip install -e "arrow/dev/archery[lint]"`
* benchmark – to run Arrow benchmarks using Archery
To install: `pip install -e "arrow/dev/archery[benchmark]"`
* docker – to run docker-compose based tasks more easily
* docker – to run docker compose based tasks more easily
To install: `pip install -e "arrow/dev/archery[docker]"`
* release – release related helpers
To install: `pip install -e "arrow/dev/archery[release]"`
Expand Down
8 changes: 4 additions & 4 deletions dev/archery/archery/crossbow/tests/fixtures/azure-linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ jobs:
- script: |
set -ex
docker -v
docker-compose -v
docker compose -v
cd arrow
docker-compose pull --ignore-pull-failures r
docker-compose build r
displayName: Docker build
docker compose pull --ignore-pull-failures r
docker compose build r
displayName: Docker build
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ tasks:
artifacts: []
params:
commands:
- docker-compose build cpp-cmake32
- docker-compose run cpp-cmake32
- docker compose build cpp-cmake32
- docker compose run cpp-cmake32
branch: ursabot-1-circle-docker-cpp-cmake32
commit: a56b077c8d1b891a7935048e5672bf6fc07599ec
_status: !TaskStatus
Expand Down
4 changes: 2 additions & 2 deletions dev/archery/archery/crossbow/tests/fixtures/crossbow-job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ tasks:
artifacts: []
params:
commands:
- docker-compose build cpp-cmake32
- docker-compose run cpp-cmake32
- docker compose build cpp-cmake32
- docker compose run cpp-cmake32
branch: ursabot-1-circle-docker-cpp-cmake32
commit: a56b077c8d1b891a7935048e5672bf6fc07599ec
_status: !TaskStatus
Expand Down
4 changes: 2 additions & 2 deletions dev/archery/archery/crossbow/tests/test_reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ def test_crossbow_export_report(load_fixture):
'https://github.com/apache/crossbow/tree/'
'ursabot-1-circle-docker-cpp-cmake32',
'circle',
{'commands': ['docker-compose build cpp-cmake32',
'docker-compose run cpp-cmake32']},
{'commands': ['docker compose build cpp-cmake32',
'docker compose run cpp-cmake32']},
'docker-tests/circle.linux.yml',
'f766a1d615dd1b7ee706d05102e579195951a61c'
]
Expand Down
30 changes: 15 additions & 15 deletions dev/archery/archery/docker/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def _execute(self, *args, **kwargs):
def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli,
using_docker_buildx):
"""
Interact with docker-compose based builds.
Interact with Docker Compose based builds.
"""
ctx.ensure_object(dict)

Expand All @@ -78,8 +78,8 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli,
"try to pass the arrow source directory explicitly.".format(src)
)

# take the docker-compose parameters like PYTHON, PANDAS, UBUNTU from the
# environment variables to keep the usage similar to docker-compose
# take the Docker Compose parameters like PYTHON, PANDAS, UBUNTU from the
# environment variables to keep the usage similar to docker compose
using_docker_cli |= using_docker_buildx
compose_bin = ("docker-compose" if using_legacy_docker_compose
else "docker compose")
Expand All @@ -98,7 +98,7 @@ def docker(ctx, src, dry_run, using_legacy_docker_compose, using_docker_cli,
@click.pass_obj
def check_config(obj):
"""
Validate docker-compose configuration.
Validate Docker Compose configuration.
"""
# executes the body of the docker function above which does the validation
# during the configuration loading
Expand All @@ -113,7 +113,7 @@ def check_config(obj):
@click.pass_obj
def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures):
"""
Execute docker-compose pull.
Execute docker compose pull.
"""
compose = obj['compose']

Expand Down Expand Up @@ -143,7 +143,7 @@ def docker_pull(obj, image, *, pull_leaf, ignore_pull_failures):
@click.pass_obj
def docker_build(obj, image, *, force_pull, use_cache, use_leaf_cache):
"""
Execute docker-compose builds.
Execute Docker Compose builds.
"""
compose = obj['compose']

Expand Down Expand Up @@ -195,7 +195,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build,
build_only, use_cache, use_leaf_cache, resource_limit,
volume):
"""
Execute docker-compose builds.
Execute Docker Compose builds.
To see the available builds run `archery docker images`.
Expand All @@ -207,7 +207,7 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build,
# execute the builds but disable the image pulling
archery docker run --no-cache conda-python
# pass a docker-compose parameter, like the python version
# pass a Docker Compose parameter, like the python version
PYTHON=3.12 archery docker run conda-python
# disable the cache only for the leaf image
Expand Down Expand Up @@ -265,15 +265,15 @@ def docker_run(obj, image, command, *, env, user, force_pull, force_build,
help='Docker repository password')
@click.pass_obj
def docker_compose_push(obj, image, user, password):
"""Push the generated docker-compose image."""
"""Push the generated Docker Compose image."""
compose = obj['compose']
compose.push(image, user=user, password=password)


@docker.command('images')
@click.pass_obj
def docker_compose_images(obj):
"""List the available docker-compose images."""
"""List the available Docker Compose images."""
compose = obj['compose']
click.echo('Available images:')
for image in compose.images():
Expand All @@ -283,14 +283,14 @@ def docker_compose_images(obj):
@docker.command('info')
@click.argument('service_name')
@click.option('--show', '-s', required=False,
help="Show only specific docker-compose key. Examples of keys:"
help="Show only specific docker compose key. Examples of keys:"
" command, environment, build, dockerfile")
@click.pass_obj
def docker_compose_info(obj, service_name, show):
"""Show docker-compose definition info for service_name.
"""Show Docker Compose definition info for service_name.
SERVICE_NAME is the name of the docker service defined on
the docker-compose. Look at `archery docker images` output for names.
SERVICE_NAME is the name of the docker service defined in
docker-compose.yml. Look at `archery docker images` output for names.
"""
compose = obj['compose']
try:
Expand All @@ -299,6 +299,6 @@ def docker_compose_info(obj, service_name, show):
click.echo(f'Service name {service_name} could not be found', err=True)
sys.exit(1)
else:
click.echo(f'Service {service_name} docker-compose config:')
click.echo(f'Service {service_name} Docker Compose config:')
output = "\n".join(compose.info(service, show))
click.echo(output)
13 changes: 7 additions & 6 deletions dev/archery/archery/docker/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ def _read_config(self, config_path, compose_bin):
'`x-hierarchy`'.format(name)
)

# trigger docker-compose's own validation
# trigger Docker Compose's own validation
if self.using_docker:
compose = Docker()
args = ['compose']
Expand All @@ -142,7 +142,7 @@ def _read_config(self, config_path, compose_bin):
stderr=subprocess.PIPE, stdout=subprocess.PIPE)

if result.returncode != 0:
# strip the intro line of docker-compose errors
# strip the intro line of docker compose errors
errors += result.stderr.decode().splitlines()

if errors:
Expand Down Expand Up @@ -203,14 +203,15 @@ def formatdict(d, template):
)
msg = (
"`{cmd}` exited with a non-zero exit code {code}, see the "
"process log above.\n\nThe docker-compose command was "
"process log above.\n\nThe {bin} command was "
"invoked with the following parameters:\n\nDefaults defined "
"in .env:\n{dotenv}\n\nArchery was called with:\n{params}"
)
raise RuntimeError(
msg.format(
cmd=' '.join(e.cmd),
code=e.returncode,
bin=self.bin,
dotenv=formatdict(self.config.dotenv, template=' {}: {}'),
params=formatdict(
self.config.params, template=' export {}={}'
Expand Down Expand Up @@ -406,10 +407,10 @@ def run(self, service_name, command=None, *, env=None, volumes=None,
cmd = service.get('command', '')
if cmd:
# service command might be already defined as a list
# on the docker-compose yaml file.
# in docker-compose.yml.
if isinstance(cmd, list):
cmd = shlex.join(cmd)
# Match behaviour from docker compose
# Match behaviour from Docker Compose
# to interpolate environment variables
# https://docs.docker.com/compose/compose-file/12-interpolation/
cmd = cmd.replace("$$", "$")
Expand All @@ -418,7 +419,7 @@ def run(self, service_name, command=None, *, env=None, volumes=None,
# execute as a plain docker cli command
self._execute_docker('run', '--rm', *args)
else:
# execute as a docker-compose command
# execute as a docker compose command
args.append(service_name)
if command is not None:
args.append(command)
Expand Down
2 changes: 1 addition & 1 deletion dev/tasks/python-wheels/github.linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
archery docker run python-wheel-manylinux-test-imports
archery docker run python-wheel-manylinux-test-unittests
# Free-threaded wheels need to be tested using a different docker-compose service
# Free-threaded wheels need to be tested using a different Docker Compose service
- name: Test free-threaded wheel
if: |
'{{ python_abi_tag }}' == 'cp313t'
Expand Down
Loading

0 comments on commit 736d706

Please sign in to comment.