docker plugins

[EugenMayer]

what do those docker-compose and docker-machine plugins offer as functionality?
Thanks for your work!

[bradeac]

Hello Eugen,

I can share with you these screenshots so you can have an idea what these plugins can do. If you have any more questions, don't hesitate to ask !

[EugenMayer]

Great, that explains a lot! So you basically try to test the start of a specific compose stack and debootstrap. Do you use those to e..g run integration tests / UAT or are there just to see "if the stack starts, all good", so smoke kind of tests?

[thomsch98]

Hi Eugen,

Camil asked me to answer your questions, because I initiated those plugins.

The plugins are are used in simple smoke tests (deployment, blue green, ...) and as part of slightly more complex pipelines (integration tests, demo deployments).

When we established a working "docker" pipeline to create and push Docker-images, the docker-compose plugin was a natural step because we use docker-compose quite a lot (ease of networking, volumes, logging ...). Scaling and the new "bundle" feature might be one of the next/last extensions to that plugin (no fixed plans for now).

For development and testing, we use the docker-machine plugin to automatically (re-)create clean slate virtual machines on vsphere (ESXi) or setup pre-created VMs at our hoster via "generic driver".

In the long run, we might switch to other tools as rancher, kubernetes or the like. Integrating/connecting our pipelines and processes with tools already providing monitoring/(auto-)scaling/orchestration, seems more efficient than trying to reinvent the wheel.

Regards

Thomas

[EugenMayer]

Well we are using rancher for production already, but sill run baremetals like proxmox for some of the other tools.

We are using docker-compose heavily in development also, also with scaffolding using e.g. the docker-compose gem or http://docker-sync.io - but thats a different topic.

Why i was asking in general is, that gocd-agents are basically not capable of doing builds in general beside you install all this build tools you most probably already have on one of the images you use in development (with compose) - so i was thinking about using compose / docker to bootstrap a container in a pipeline, so that builds can happen in that container.

In the meantime, i happend to talk to the gocd people and they are already working on such a feature in general, called "elastic containers", so that agents basically bootstrap a container per task ( or job, idk ) and you setup a dockerfile to define your build dependencies so the builds will work just fine.

Back to your plugins, i think that using this plugin to smoke-test plugins can have quiet some issues, doesnt it. If you think about the natural issues with starting orders, you usually end up having a scaffolding thor/rake task to wait for specific sockets like DB to be accessable, before you can continue bootstrapping the app. Sure, we also use consul in some projects to ease this up using service-auto-discovery, but still, docker-compose ofter does not soley solve bootstrapping those containers properly to really do a "smoke test".

What about your experiences on that field? Or is the docker-compose stack rather simple so that you do not run into those raise conditions?

Greating to you at Freiburg, from Freiburg :)

[thomsch98]

Hi Eugen,

we are using the plugins even inside a sole Docker-based env (e.g. my laptop).

Server and agent(s) are Docker-containers themselves, the plugins inside the container pull/build/pushe Docker images AND can be used to RUN created/pulled containers to create other artifacts:
One of our pipelines starts a nodejs container to run some nodejs script that is treated as an artifact.
Basically, this is our approach to establish "Docker-only" agents.

We haven't decided if we REALLY end up using completely dockerized agents (I DO hope so) or if we are going to use conventional gocd agents with Docker at the ready as the "only build-tool" (no installation of golang, dotnetcore, c++ on the VM) - somewhat identical to elastic containers but implemented with our own task-plugins.

Any startup-sequence problems (there were quite a few...) of docker-services usually have been addressed by enabling our apps to cope with "unavailability of dependencies" by themselves, or by using Jason Wilders dockerize (https://github.com/jwilder/dockerize).