Updates handling of scale/replicas parameter in CLI and compose file

[me-coder]

Purpose

Addresses issue: #267

Contributor Checklist:

If this PR adds a new feature that improves compatibility with docker-compose, please add a link
to the exact part of compose spec that the PR touches.

This PR attempts to address missing (the code to accommodate this was already present, just broken) --scale option for podman-compose up:
https://docs.docker.com/reference/cli/docker/compose/up/#options

All changes require additional unit tests.

Testing

Test file:

name: podman-compose

services:
  app:
    image: docker.io/library/busybox:latest
    tty: true
    # scale: 3
    # deploy:
    #   mode: replicated
    #   replicas: 2

  app1:
    image: docker.io/library/busybox:latest
    tty: true
    depends_on:
      - app

  app2:
    image: docker.io/library/busybox:latest
    tty: true

  app3:
    image: docker.io/library/busybox:latest
    tty: true

With compose file:

$ python ./podman_compose.py -f ./test.yml up -d
1283f00b651460df07514d3be6d01e0ea76529782ead0292c977077b72229937
d3b036808fff09a6e107c3ba769ad03a2c024d148581a24bdd5e3128b952d976
663cb01d16c4e7194145b63a306d6e225f7c05b1ff97103ae8e2df57144233b7
8dd9e7f5588ec2d1ecd16565cdf371be8fce87442f902e48bec7bf97276026b4

$ python ./podman_compose.py -f ./test.yml ps
CONTAINER ID  IMAGE                             COMMAND     CREATED        STATUS        PORTS       NAMES
d3b036808fff  docker.io/library/busybox:latest  sh          7 seconds ago  Up 7 seconds              podman-compose_app_1
663cb01d16c4  docker.io/library/busybox:latest  sh          6 seconds ago  Up 6 seconds              podman-compose_app_2
8dd9e7f5588e  docker.io/library/busybox:latest  sh          5 seconds ago  Up 5 seconds              podman-compose_app_3

$ python ./podman_compose.py -f ./test.yml down
podman-compose_app_1
podman-compose_app_3
podman-compose_app_2
podman-compose_app_3
podman-compose_app_2
podman-compose_app_1
1283f00b651460df07514d3be6d01e0ea76529782ead0292c977077b72229937
podman-compose_default

Scale existing services using CLI:

$ python ./podman_compose.py -f ./test.yml up -d --scale=3
5a76ea112148b71be792cd4c9e77f7b505cbdc2432118eaa2bff8003753b1bb9
ca5b2c05a5198392d64560b63a57e8db0cf2ecdb7f1171f10e800824cc1974d3
8f835d6759a2614c59ee7d21abd0859e6cc96ffec06bdceae3dc55ab096bd488
1fbc4958fa4439d8e0349459f77f1d59026ed3414b815026992611ee8518e141

$ python ./podman_compose.py -f ./test.yml ps
CONTAINER ID  IMAGE                             COMMAND     CREATED        STATUS        PORTS       NAMES
ca5b2c05a519  docker.io/library/busybox:latest  sh          5 seconds ago  Up 5 seconds              podman-compose_app_1
8f835d6759a2  docker.io/library/busybox:latest  sh          4 seconds ago  Up 4 seconds              podman-compose_app_2
1fbc4958fa44  docker.io/library/busybox:latest  sh          3 seconds ago  Up 3 seconds              podman-compose_app_3

$ python ./podman_compose.py -f ./test.yml down
podman-compose_app_3
podman-compose_app_2
podman-compose_app_1
podman-compose_app_3
podman-compose_app_2
podman-compose_app_1
5a76ea112148b71be792cd4c9e77f7b505cbdc2432118eaa2bff8003753b1bb9
podman-compose_default

To-Do

• Scale-up with CLI works (although with errors on command line, which should be warnings). But, Scale-down doesn't work.
  => That part of code yet needs to be addressed, may be through a separate issue.
• Command down (to destroy the scaled containers/replicas) works only based on value assigned in compose yaml file. This instead, needs to be addressed to stop all running replicas.
  => Done

[p12tic]

I think we should use None to signify that no value has been supplied, and then handle the None case at the very end. Right now if explicit 1 value is passed it may be overridden by next steps.

[me-coder]

Thanks @p12tic for the review and suggestion.

I've updated the code to handle the cases:

1. --scale from the CLI up command
2. scale from compose yaml file
3. {"deploy": {"replicas": <NUM>} from compose yaml file

Was tempted to use the new fancy match-cases, but didn't to maintain compatibility with pre-Python-2.10 versions.

Do let me know if the new code is fine.

Also, I've modified the compose_down function to behave more closely to the docker compose down command:
8036f67

[p12tic]

This will way on all services to stop one by one. This is problematic in case of large number of services. One deployment I have is 40 services for example.

[me-coder]

They would stop in parallel as await yields from the compose.podman.run([], "stop", [*podman_stop_args, cnt["name"]]).
So rather than waiting for asyncio.gather to schedule the future tasks/co-routines, here they get scheduled as soon as they are passed and the control returns.

However, the concern is valid. It won't wait for all the stop tasks to finish before proceeding to await compose.podman.run([], "rm", [cnt["name"]]). I'll try to fix it. (Ideally should be upgraded to TaskGroup, but for that there has to be an upgrade mandate to Python 3.11 for compatibility reasons).

[me-coder]

Done. Updated the code to use asyncio.gather for stop and rm command tasks.

[p12tic]

No need to add (Issue: containers#267) in commit description.

[p12tic]

Overall code looks good.

We need integration tests. You've already done most of the work and provided compose file and expected output in the PR description. What's left is to automate it. Check out tests/integration for examples of how this can be done.

Updates to compose_down command to match docker compose behavior should probably live in another PR because the change is unrelated. This would make it easier to land "Updates handling of scale/replicas through CLI & compose file" commit.

Talking about Updates to compose_down command to match docker compose behavior, it also needs tests and release note (probably several, because more than one thing is fixed). Also, I would say that the commit should itself be split into several, because right now it's hard to understand what the commit is trying to achieve. It's clear that it does improve something, but it contains several improvements merged into one.