MDBF-876 - Remove BuildBot configuration code needed for systemd setup #674
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cvicentiu
approved these changes
Jan 8, 2025
|
@RazvanLiviuVarzaru Please push once you make the agreed upon changes. I'll be building on top of this one. |
--env-file behaves differently between docker-compose and docker run. 1. docker run --env-file cannot handle white-spaces and new-line Reference: moby/moby#12997 Workaround: -> used one-liners for multi-line variable values -> removed unnecessary whitespaces 2. docker run --env-file will preserve single/double quotes Reference: docker/cli#3630 Workaround: -> in validate_master script use --env-file <(sed "s/='\([^']*\)'/=\1/" $ENVFILE) to remove the single quotes -> modified .env files to be quote consistent. Single quotes for variable value. Double quotes for JSON structures inside variable value Test: - ./validate_master_cfg.sh -e PROD - ./validate_master_cfg.sh -e DEV - run docker-compose and check variable values The sample master private file was changed for two reasons: 1. If you want to run buildbot locally, MariaDB Server (docker) will run by default with --skip-name-resolve thus using 127.0.0.1 is mandatory to allow buildbot to connect to the database 2. bb-rhel9-docker entry was added because now the validation scripts load the specific Dev environment variables for which there's a different workername for the docker builder
RazvanLiviuVarzaru
force-pushed
the
feature/validate-container-environment
branch
from
4e07bd7 to
986076f
Compare
Both BuildBot environments now run in containers so we can get rid of the default="" in << os.getenv >>. To check that the variable is defined and to prevent typos use os.environ which would raise a key error if the variable does not exist. PORT is a special case, because it's defined in docker-compose generation script and is not present in the .env and .env.dev files, hence the hardcoded value in the validation script. Keep os.environ for PORT because the correct value will be passed by docker-compose during initialization (docker-compose --env-file .env up)
This was the "systemd" approach on BuildBot services for getting the PORT number. Now that both environments run in docker containers, the port variable is exposed to the container environment via docker-compose.
RazvanLiviuVarzaru
force-pushed
the
feature/validate-container-environment
branch
from
986076f to
cdcf7c8
Compare
@cvicentiu |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously when the BuildBot Production environment was running on
systemd services, we couldn't use the.envfiles defined in the project tree because they were specifically designed for usage indocker-compose.This led to hard coding variable defaults in the buildbot code, in the form of
os.getenv(#env_variable_for_containers#, default=#default_for_systemd#)In this task:
-> switch to a fail proof approach of loading variables using
os.environwhich will raise akey not found exceptionif the environment variable is not defined-> adapt the GitHub CI script
validate_mastersto load the environment variables for each environment << dev , prod >>-> remove "systemd" approach of loading PORTS for masters.
This PR was tested using:
./validate_master_cfg.sh -e {PROD,DEV}- run docker-compose with .env filesPlease see the commit messages for more details.