-
Notifications
You must be signed in to change notification settings - Fork 66
Concepts of building
Updated: 2020-12-31
- Concepts of building
This page describes the concepts, that should help by understanding the specifics of the used building process.
This page does not describe the common and well-known concepts of building Docker images.
The concepts described here are specific to this project and they help by achieving the project goals.
This page should also help by understanding how the actual implementation works.
It can help to read about the Concepts of Dockerfiles first.
There are actually two different building process concepts. The first one is about building a single image and the other one is about building and publishing sets of images into Docker Hub repositories.
Building a single image is generally a two step process.
In the first step a helper image is built. It is used only for getting the current values of the version stickers. The concept of version stickers is described on the page Concepts of Dockerfiles.
The value of the verbose version sticker is then compared with the value of the previously published build of the image. If there is no change, then there is no need to refresh the image and the other parts of the building pipeline can be skipped.
For this concept to work, there is a need of persisting the verbose version sticker values somewhere outside the image. The current implementation of the pipeline uses GitHub gists.
Strictly speaking, building the helper image is only necessary, if the final image needs to be labeled with the up-to-date value of the short version sticker. Otherwise the first building step can be skipped completely.
On the other hand, only this step is needed for simple checking if refreshing the image is necessary.
If the value of the verbose version sticker has changed since the last release, then the final image is built in the second step.
This time also its labels created and version-sticker are set to the current up-to-date values from the first step.
Building and publishing images involves two kinds of Docker Hub repositories that are closer described below.
The reasons for this decision include the following:
- Having more freedom for experimenting by developing new images
- Possibility to use repositories of different visibility (private or public)
- Docker Hub does not support renaming of repositories
- Dockerfiles and README files published automatically on Docker Hub are not always the correct ones
- Automatic publishing of README files on Docker Hub is not flexible
In practice, both repository roles could be played by a single Docker Hub repository. This would require a few adjustments in the implementation, but it is possible. However, naming of images and tags becomes to be a bigger challenge then.
There is a single repository, which is used for building all the images. It is referenced as a builder repository. This repository is bound to the GitHub repository containing the necessary resources.
The builder repository repository is not intended for public use and it could be kept private. The related GitHub repository could be also kept private.
The Dockerfile and the README file, that are automatically published by the auto-builder on Docker Hub, are irrelevant.
Strictly speaking, you could go also without the builder repository, if you would prefer to build the images only locally and push them into the deployment repositories from the local stage. However, this is not the primary scenario of this project and it will not be described here.
The final images are pushed into one or more deployment repositories on Docker Hub.
The names of these repositories and the names of the image tags are independent from the ones in the builder repository.
Deployment repositories are generally not bound to the GitHub repository used by the builder repository.
The final README files in deployment repositories are explicitly published after pushing the images.
They are no Dockerfiles in deployment repositories. The links to them should be included in the README files.
Achieving some of the project goals and supporting the concept of single image building requires independent stores for persisting the required metadata.
There should be two independent metadata stores - one for the builder repository and one for the deployment repositories. There are conceptually the same and generally contain the same kinds of metadata. However, there could also be additional metadata in both of them.
Both metadata stores could be private or public.
Any persistent store could be used, but the current implementation uses GitHub gists.
Currently there the following kinds of metadata stored in the gists:
- JSON endpoint for the
createdbadge, which is the time and date when the final image has been built - JSON endpoint for the
version-stickerbadge, which is the short version sticker value - content of the verbose version sticker, which is used by the building pipeline and which is also linked with the
version-stickerbadge in the README file
It should be understood, that the gist belonging to the deployment repositories merely stores the metadata (e.g. badge endpoints) of the published images. It is maintained by the building pipeline, but it does not participate in decision making if an image needs to be refreshed. Only the verbose version sticker values from the gist belonging to the builder repository are taken into consideration.
The following concepts allow better separation of concerns by implementing the building pipeline.
The building branch is roughly the same concept as the source branch of the auto-builder on Docker Hub.
However, the source branch on Docker Hub means always the used git branch on GitHub.
By building locally there is no need for such tight binding.
The building branch is just a concept of a set of building scenarios.
This concept is related to the concept of features, because it means a mix of features of the particular image build.