Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update deployment container script #396

Merged
merged 33 commits into from
Jul 5, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
07b7d47
Build images workflows pushes to latest tag as well
patrick-5546 May 26, 2024
7f6aded
Make specifying tag required
patrick-5546 May 26, 2024
f302948
Remove unneeded parts of run-tests script
patrick-5546 May 26, 2024
a077334
Start revamping start container script to use docker compose
patrick-5546 May 26, 2024
259e7ed
Switch user and workdir
patrick-5546 May 27, 2024
a573598
Merge remote-tracking branch 'origin/main' into patrick-5546/update-d…
patrick-5546 May 27, 2024
5925e3a
Use long argument names
patrick-5546 Jun 2, 2024
d196355
Use dev version of website
patrick-5546 Jun 2, 2024
4d2d57e
Rebuild on changes to dockerfiles
patrick-5546 Jun 2, 2024
c4819e6
Run software
patrick-5546 Jun 2, 2024
f163c82
Update argument usage
patrick-5546 Jun 2, 2024
a882ab1
Do not publish base and local-base images
patrick-5546 Jun 2, 2024
5391eaf
Improve argument parsing
patrick-5546 Jun 2, 2024
71bc4b6
Add clean and reset arguments
patrick-5546 Jun 2, 2024
779249c
Update the deployment readme
patrick-5546 Jun 2, 2024
bbd23d4
Can run without internet
patrick-5546 Jun 2, 2024
b2078ad
Ignore all node modules in web gitignore
patrick-5546 Jun 2, 2024
ab4276b
Add volume for web tests node modules
patrick-5546 Jun 2, 2024
287841c
clean and reset before running commands not after
patrick-5546 Jun 2, 2024
0aad7c9
Downgrade mongodb to be compatible with rpi
patrick-5546 Jun 2, 2024
225cd62
Remove deprecated build option
patrick-5546 Jun 2, 2024
c3a2c2d
Debugging failing CI
patrick-5546 Jun 2, 2024
2f91e09
Give user ownership of ros workspace
patrick-5546 Jun 2, 2024
5d3a2af
Add sudo to purge command
patrick-5546 Jun 2, 2024
eedc966
Merge remote-tracking branch 'origin/main' into patrick-5546/update-d…
patrick-5546 Jun 3, 2024
f8f05bd
Fiddle around with user permissions
patrick-5546 Jun 3, 2024
79b6736
Source in run-tests
patrick-5546 Jun 4, 2024
f40e564
Cleanup
patrick-5546 Jun 4, 2024
77e15d5
Switch to prod version of website
patrick-5546 Jun 4, 2024
47aed7c
Create run_software.sh and cleanup start_containers.sh
patrick-5546 Jun 5, 2024
589479e
Make start_containers.sh compatible with mac
patrick-5546 Jun 5, 2024
339489e
Try specifying user in docker compose file
patrick-5546 Jun 6, 2024
be88eb3
Merge branch 'main' into patrick-5546/update-deployment-container
patrick-5546 Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,7 @@ COPY --chown=${USERNAME}:${USERNAME} config ${HOME}
# && apt-get autoremove -y \
# && apt-get clean -y \
# && rm -rf /var/lib/apt/lists/{apt,dpkg,cache,log} /tmp/* /var/tmp/*

# set user and working directory
USER ${USERNAME}
WORKDIR ${ROS_WORKSPACE}
16 changes: 1 addition & 15 deletions .devcontainer/base-dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,23 +62,9 @@ If you want to push this to GitHub (instead of using the slow Build Images workf

1. Login to the GitHub container registry: [Authenticating with a personal access token (classic)](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic)

2. Push each image, replacing `<tag>` with the desired tag in the commands below
2. Push the image, replacing `<tag>` with the desired tag in the command below

```
docker buildx build . \
--file base-dev.Dockerfile \
--tag ghcr.io/ubcsailbot/sailbot_workspace/base:<tag> \
--platform linux/arm64,linux/amd64 \
--builder sailbot \
--target base \
--push
docker buildx build . \
--file base-dev.Dockerfile \
--tag ghcr.io/ubcsailbot/sailbot_workspace/local-base:<tag> \
--platform linux/arm64,linux/amd64 \
--builder sailbot \
--target local-base \
--push
docker buildx build . \
--file base-dev.Dockerfile \
--tag ghcr.io/ubcsailbot/sailbot_workspace/dev:<tag> \
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ services:

# URL: mongodb://localhost:27017
mongodb:
image: mongo:7
image: mongo:${MONGO_TAG:-7}
restart: unless-stopped
volumes:
- mongodb-data:/data/db:delegated
Expand Down
33 changes: 5 additions & 28 deletions .github/workflows/build-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,9 @@ on:
workflow_dispatch:
inputs:
tag:
description: 'Tag of the base, local-base, and dev images'
required: false
description: 'Tag of the Docker image'
required: true
type: string
default: 'main'

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand All @@ -34,37 +33,15 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.PAT_TOKEN }}

- name: Build and push base
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:.devcontainer/base-dev"
file: base-dev.Dockerfile
push: true
tags: ghcr.io/ubcsailbot/sailbot_workspace/base:${{ inputs.tag }}
platforms: linux/arm64,linux/amd64
cache-from: type=gha
cache-to: type=gha,mode=max
target: base

- name: Build and push local-base
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:.devcontainer/base-dev"
file: base-dev.Dockerfile
push: true
tags: ghcr.io/ubcsailbot/sailbot_workspace/local-base:${{ inputs.tag }}
platforms: linux/arm64,linux/amd64
cache-from: type=gha
cache-to: type=gha,mode=max
target: local-base

- name: Build and push dev
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:.devcontainer/base-dev"
file: base-dev.Dockerfile
push: true
tags: ghcr.io/ubcsailbot/sailbot_workspace/dev:${{ inputs.tag }}
tags: |
ghcr.io/ubcsailbot/sailbot_workspace/dev:${{ inputs.tag }}
ghcr.io/ubcsailbot/sailbot_workspace/dev:latest
platforms: linux/arm64,linux/amd64
cache-from: type=gha
cache-to: type=gha,mode=max
40 changes: 8 additions & 32 deletions scripts/deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,6 @@ Deploying our software to our autonomous sailboat's main computer.

## Scripts

### `start_container.sh`

Runs the [`base`](https://github.com/UBCSailbot/sailbot_workspace/blob/main/.devcontainer/base-dev/base-dev.Dockerfile)
image. A new container is created every time this is run. The default container name is `sailbot_deployment_container`.
Container names are unique, so if you want to use multiple deployment containers (e.g., from different branches)
you will have to update the variable `CONTAINER_NAME` in the script.

Usage:

- Runs the base image used by the Dev Container by default: `./start_container.sh`
- Run a specific version of the base image by specifying its ID: `./start_container.sh <IMAGE_ID>`

### `setup_boot.sh`

Configures programs and scripts that need to run when the main computer boots. Only needs to be run once unless the
Expand All @@ -24,27 +12,15 @@ renaming or moving the file.

Usage:

- Must be run as root
- `sudo ./setup_boot.sh`
- Must be run as root: `sudo ./setup_boot.sh`

## Deployment container commands
### `start_containers.sh`

- Exit out of a container: `exit`
- Start an existing container: `docker start -ia <container name>`
- Delete an existing container: `docker rm <container name>`
- Find the container ID of a container: `docker ps -a`
Runs our Docker Compose files.

## Deploy software

> These commands are run in the in the root directory of this repository

1. Run the setup script: `./scripts/setup.sh`
2. Run the build script with the quick build flag: `./scripts/build.sh -q`

## Develop software

The deployment container isn't intended for development, but if you discover a bug and want to quickly push a fix:
Usage:

1. Fix the issue in a terminal ***outside*** the deployment container
2. Run the software to verify your fix in a terminal ***inside*** the deployment container
3. Commit and push your fix in a terminal ***outside*** the deployment container
- Runs the global launch file by default: `./start_containers.sh`
- Add the `--website` argument to additionally run the website container
- Add the `--interactive` argument to manually run commands in the sailbot workspace container
- Add the `--help` argument to see all available arguments
51 changes: 0 additions & 51 deletions scripts/deployment/start_container.sh

This file was deleted.

128 changes: 128 additions & 0 deletions scripts/deployment/start_containers.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
#!/bin/bash

## CLI ARGUMENTS ---

# default argument values
WEBSITE_ARG=""
INTERACTIVE=false
CLEAN=false
RESET=false

# function to display usage help
usage() {
echo "Usage: $0 [--website] [--interactive] [--clean] [--reset]"
echo " --website If set, runs the website container"
echo " --interactive If set, run commands inside the sailbot workspace container interactively"
echo " --clean If set, removes containers after running"
echo " --reset If set, removes containers and volumes after running"
exit 1
}

# parse command-line options
while getopts ":-:" opt; do
case "${opt}" in
-)
case "${OPTARG}" in
website)
WEBSITE_ARG="--file .devcontainer/website/docker-compose.website.yml"
patrick-5546 marked this conversation as resolved.
Show resolved Hide resolved
;;
interactive)
INTERACTIVE=true
;;
clean)
CLEAN=true
;;
reset)
RESET=true
;;
*)
usage # Show usage if argument is not recognized
;;
esac
;;
*)
usage # Show usage if option is unknown
;;
esac
done

# check for any remaining arguments after option parsing
if [ "$OPTIND" -le "$#" ]; then
usage
fi

## HELPER VARIABLES AND FUNCTIONS ---

# get absolute path to workspace root in host OS
SCRIPT_DIR="$(dirname "$(readlink --canonicalize "$0")")"
HOST_WORKSPACE_ROOT="$SCRIPT_DIR/../.."

# common arguments for docker compose commmands
MONGO_TAG="7"
PROJECT_NAME="deployment"
DOCKER_COMPOSE_ARGS="docker compose --project-name $PROJECT_NAME --file .devcontainer/docker-compose.yml $WEBSITE_ARG"

# function to pull the first FROM image from a specified Dockerfile
pull_from_image() {
# Check if Dockerfile path was provided as an argument
if [[ -z "$1" ]]; then
echo "No Dockerfile path specified."
return 1 # Return with error status
fi

# The path to the Dockerfile is the first argument to the function
local dockerfile_path="$1"

# Extract the image from the Dockerfile
local image=$(grep '^FROM' "$dockerfile_path" | head -n 1 | awk '{print $2}')

# Check if the image variable is not empty
if [[ -n "$image" ]]; then
echo "Pulling Docker image: $image"
docker pull "$image"
else
echo "No image found in Dockerfile at $dockerfile_path."
return 1 # Return with error status
fi
}


## RUN COMMANDS ---

# run commands in the workspace's root directory
cd $HOST_WORKSPACE_ROOT

# pull images if connected to the internet
if wget -q --spider --timeout=1 http://google.com; then
pull_from_image .devcontainer/Dockerfile
pull_from_image .devcontainer/website/website.Dockerfile
docker pull mongo:$MONGO_TAG
fi

# start containers
MONGO_TAG=$MONGO_TAG $DOCKER_COMPOSE_ARGS up --build --detach --pull never

# run commands inside sailbot workspace container
if [[ "$INTERACTIVE" = true ]]; then
$DOCKER_COMPOSE_ARGS exec --interactive --tty sailbot-workspace /bin/bash
else
$DOCKER_COMPOSE_ARGS exec --no-TTY sailbot-workspace /bin/bash -c "\
source /opt/ros/\$ROS_DISTRO/setup.bash && \
./scripts/setup.sh && \
./scripts/build.sh && \
source ./install/local_setup.bash && \
ros2 launch src/global_launch/main_launch.py"
fi

# stop containers
$DOCKER_COMPOSE_ARGS stop

# remove containers
if [[ "$CLEAN" == "true" ]] || [[ "$RESET" == "true" ]]; then
$DOCKER_COMPOSE_ARGS down
fi

# remove volumes
if [[ "$RESET" = true ]]; then
docker volume ls -q | grep "^${PROJECT_NAME}_" | xargs -r docker volume rm
fi
3 changes: 1 addition & 2 deletions scripts/run-tests.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#!/bin/bash
set -e

source /opt/ros/${ROS_DISTRO}/setup.bash
./scripts/setup.sh
./scripts/build.sh RelWithDebInfo OFF # Do not run static analysis or linting
./scripts/build.sh
./scripts/test.sh
5 changes: 4 additions & 1 deletion scripts/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,8 @@ for DIR in $ROS_WORKSPACE/src/*; do
done

sudo apt-get update
rosdep update --rosdistro $ROS_DISTRO
if wget -q --spider --timeout=1 http://google.com; then
# only run if connected to the internet
rosdep update --rosdistro $ROS_DISTRO
fi
rosdep install --from-paths src --ignore-src -y --rosdistro $ROS_DISTRO
Loading