Docker/ansible provisioning process

[pedro-psb]

Hello everyone,

I was trying to understand the provisioning process of these plugged tools. I'll share what I could come up with at the moment.

• up should keep an Ansible playbook (is that it? I've never really used ansible) that aggregates provisioning instructions from all plugged tools.
• this playbook is used in a Dockerfile managed by up to build a provisioned-up-img.
• collected commands are called internally with docker run (...) provisioned-up-img bash -c "my-tool"

Sample process:

$ pip install up
$ up ls
Nothing installed yet.

$ pip install up-git # git plugin for up
$ up ls --verbose
[INFO] New plugin detected: "git"
[INFO] Registering "git" metadata...
[INFO] Updating ansible provisioning instructions...
[INFO] Building up-to-date docker image...
[INFO] Ready!
git

$ up git status --verbose
[INFO] Calling: docker run provisioned-up-img bash -c "git status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

This is still a little fuzzy to me, please do share other ideas, considerations, and alternatives.

To help clarify this process, it would be helpful to make some manual experiments isolated from the up-cli context, like building a basic setup with ansible/docker to provision a specific tool and then running it with a raw docker command.

[faermanj]

I believe that the simplest thing would be to map the command to an image and run it with the correct settings (variables, volumes, etc)

For example, if the user runs up ansible testservers – list -i ansible_hosts it would:

• load the appropriate plugin for that command (ansible)
• invoke the plugin for the appropriate image to run
• invoke the plugin for the settings
• run the command in the container (likely using docker-in-docker, because the up-cli itself should run in a container)

That would be for actual binaries. For solutions, it would also invoke the plugin to map the solution to the actual commands, but I believe that is not now.

[pedro-psb]

Okay, so for each plugin we would have one image? I was thinking about up maintaining one image and plugins providing aggregations on top of it.

One disadvantage is that the image could become too complex. On the other hand, all binaries would be available under the same image, so it would be easier to compose commands using any available binary.

Draft

Here is a draft of how I imagine we would build images, in any of those cases:

1:1 plugin-img

Images

up-git-img

# provided by `up-git`
FROM fedora:latest
RUN dnf -y update && dnf -y install git

up-openshift-img

# provided by `up-openshift`
FROM fedora:latest
RUN dnf -y update && dnf -y install openshift

Run Map

docker run (...) up-git-img bash -c "git --version"
docker run (...) up-openshift-img bash -c "openshift --version"

n:1 plugin-img

Images

up-git-fragment

# (provided by `up-git`
RUN dnf -y update && dnf -y install git

up-openshift-fragment

# (provided by `up-openshift`
RUN dnf -y update && dnf -y install openshift

up-global-img

# final aggregation
FROM fedora:latest
{{ up-git-fragment }}
{{ up-openshift-fragment }}

Run Map

docker run (...) up-global-img bash -c "git --version"
docker run (...) up-global-img bash -c "openshift --version"

[pedro-psb]

run the command in the container (likely using docker-in-docker, because the up-cli itself should run in a container)

Why it should run inside a container? If it's about system-python isolation there are some non-exclusive alternatives, such as pipx or an installation step that uses a dedicated venv.

[faermanj]

The main reason for running as a container is to avoid platform-specific issues, making it easy to run anywhere.
Also kind of following the design of the AWS CLI

[faermanj]

I tried to put these ideas to code and recorded this brief demo:
https://www.loom.com/share/8cd88f4c9829459fb8626b958c106b43?sid=6a058a9d-3c22-4dfa-b2d3-e64a5beb79b0

let me know what you think :)

[pedro-psb]

This looks like a solid starting point and it is clearer how it should work in general.

One thing I can't fully visualize yet is how the tool should integrate with a project dev environment. For example, to use it in a web app project, would I need to configure the volumes and network so the project is run inside the specific tool-container (inside the up-container), or that wouldn't be necessary?

We may also need some work with auth so we don't need to authenticate on every run (that part is even less clear for me. E.g, I guess this would apply in the case someone wants to auth with an ssh-key that already lives on the host machine.

[fabiobarkoski]

Yeah, I agree with Pedro on how good is looking and in the auth question, a .conf file is a good idea?
Also, the up_cli will have flags options? If yes, how it will be the differentiation between the flags in up_cli to another CLI's.