Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caio new da #1958

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .github/workflows/review-exercises.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
name: Review Exercises
on:
pull_request:
paths:
- 'students/**/*'

jobs:
review-exercises:
name: Review Exercises
runs-on: ubuntu-latest
env:
REVIEWER_FOLDER: reviewer
STUDENTS_FOLDER: students
steps:
- uses: actions/checkout@v3

- uses: dorny/paths-filter@v2
id: changes
with:
list-files: json
filters: |
git03:
- '${{ env.STUDENTS_FOLDER }}/**/git03/**/*'
git04:
- '${{ env.STUDENTS_FOLDER }}/**/git04/**/*'

# - name: Debug
# run: |
# GITHUB_ACTOR=${{ github.actor }}
# EXERCISE=$(echo "${{ toJson(steps.changes.outputs.changes) }}" | jq -r '.[0]')
# LATEST_VERSION=$(cat ${{ env.REVIEWER_FOLDER }}/exercises/$EXERCISE/config.json | jq -r '.version')
# STUDENT_VERSION=$(cat students/${{ github.actor }}/exercises/$EXERCISE/config.json | jq -r '.version')

# echo ${{ toJson(steps.changes.outputs.changes) }}
# echo "${{ toJson(steps.changes.outputs.changes) }}" | jq '. | length'
# echo EXERCISE=$EXERCISE
# echo LATEST_VERSION=$LATEST_VERSION
# echo STUDENT_VERSION=$STUDENT_VERSION
# echo GITHUB_ACTOR=$GITHUB_ACTOR

- name: Check if student username exists
run: |
GITHUB_ACTOR=${{ github.actor }}
if [[ ! -d "${{ env.STUDENTS_FOLDER }}/$GITHUB_ACTOR" ]];then
echo "::error title=Student not found::Your Github username wasn't found under students/. Please make sure to use your Github username as student_name.";
exit 1;
fi;

- name: Check if multiple exercises have been submited
run: |
EXERCISES_CHANGED=$(echo "${{ toJson(steps.changes.outputs.changes) }}" | jq '. | length')
if [ $EXERCISES_CHANGED -gt 1 ];then
echo "::error title=Multiple exercises found::Please submit only 1 exercise at a time.";
exit 1;
fi;

- name: Check if exercise is on latest version
id: check
run:
EXERCISE=$(echo "${{ toJson(steps.changes.outputs.changes) }}" | jq -r '.[0]')
LATEST_VERSION=$(cat ${{ env.REVIEWER_FOLDER }}/exercises/$EXERCISE/config.json | jq -r '.version')
STUDENT_VERSION=$(cat students/${{ github.actor }}/exercises/$EXERCISE/config.json | jq -r '.version')

echo LATEST_VERSION=$LATEST_VERSION
echo STUDENT_VERSION=$STUDENT_VERSION

if [ $LATEST_VERSION != $STUDENT_VERSION ];then
echo "::error title=Not latest version::Student exercise $EXERCISE version is $STUDENT_VERSION, not the latest ($LATEST_VERSION). Please update and submit again.";
exit 1;
fi;

echo "exercise=$EXERCISE" >> $GITHUB_OUTPUT

echo "version=$STUDENT_VERSION" >> $GITHUB_OUTPUT

- name: ${{ steps.check.outputs.exercise }}/${{ steps.check.outputs.version }} - Review exercise
run: |
IMAGE=devopsacademyau/reviewer:${{ steps.check.outputs.exercise }}-${{ steps.check.outputs.version }}
docker run \
--name reviewer \
-v ${{ github.workspace }}/${{ env.STUDENTS_FOLDER }}/${{ github.actor}}/exercises/${{ steps.check.outputs.exercise }}/:/exercise \
$IMAGE

if [ $? -ne 0 ];then

exit 1;
fi;

docker logs reviewer | grep "SUMMARY:" > output
while read line; do
echo "${line}" | sed 's/SUMMARY: //' >> $GITHUB_STEP_SUMMARY
done < output
41 changes: 41 additions & 0 deletions .github/workflows/reviewer-build-push.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
on:
workflow_call:
inputs:
exercise:
required: true
type: string

jobs:
reviewers-build-push:
name: Build ${{ inputs.exercise }}/${{ github.event_name }}
runs-on: ubuntu-latest
env:
IMAGE_NAME: devopsacademyau/reviewer
steps:
- uses: actions/checkout@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build tag name
id: tag
run: |
VERSION=$(cat reviewer/exercises/${{ inputs.exercise }}/config.json | jq -r .version)
SUFFIX=$(if [ ${{ github.event_name }} = "pull_request" ]; then echo "-pr-${{ github.event.number }}"; else echo ""; fi;)
echo "tag=${{ env.IMAGE_NAME }}:${{ inputs.exercise }}-$VERSION$SUFFIX" >> $GITHUB_OUTPUT

- name: Build and push
uses: docker/build-push-action@v3
with:
push: true
context: ./reviewer/exercises/${{ inputs.exercise }}
tags: ${{ steps.tag.outputs.tag }}
42 changes: 42 additions & 0 deletions .github/workflows/reviewer-setup.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Reviewers Setup
on:
pull_request:
paths:
- 'reviewer/exercises/**/*'
push:
branches:
- 'master'
- 'main'
paths:
- 'reviewer/exercises/**/*'

jobs:
reviewers-setup:
name: Get list of Reviewers to be build
runs-on: ubuntu-latest
outputs:
exercises: ${{ steps.changes.outputs.changes }}
steps:
- uses: actions/checkout@v3

- uses: dorny/paths-filter@v2
id: changes
with:
list-files: json
# better to list individually to have a cleaner integration with matrix strategy when calling build jobs
filters: |
git03:
- 'reviewer/exercises/git03/**/*'
git04:
- 'reviewer/exercises/git04/**/*'

reviewers-build-push:
name: Build&Push Reviewer
needs: [reviewers-setup]
uses: devopsacademyau/academy/.github/workflows/reviewer-build-push.yaml@caio-new-da
strategy:
matrix:
exercises: ${{ fromJson(needs.reviewers-setup.outputs.exercises) }}
with:
exercise: ${{ matrix.exercises }}
secrets: inherit
26 changes: 26 additions & 0 deletions exercises/git04/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## Secrets (c01-git04)

Perform the following commands:
1. In the same repository of the previous exercise:
2. Add a file called `my_env.txt` that contains
```
SERVICE_NAME=account-management
ENVIRONMENT=prod
PASSWORD=pass1234
```
1. Commit it to your local repository
2. Check the log with `git log`

**Questions**

1. Let's suppose you remotely pushed the `my_env.txt` file above. A colleague asks you to remove this information from Git. What's your colleague worried about?
2. If you modify the file in your workspace, then commit and push it, will it be enough to erase this password information from the repository? (It's not). Why?
3. If you delete the file and push it, then create a new one with the rest of the information, is it enough? (It's not). Why?
4. How to fix this? How do you remove something from Git history when it is in the remote repository?
5. Which commands would you use? Explain what the command does.

## Submit a PR with the following files

> Remember to follow the instructions on [how to submit a PR here](/README.md#exercises)

- **README.md**: copy from file [ANSWER.md](ANSWER.md), answering the questions above. Include details and commands used.
3 changes: 3 additions & 0 deletions exercises/git04/my_env.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
SERVICE_NAME=account-management
ENVIRONMENT=prod
PASSWORD=pass1234
41 changes: 41 additions & 0 deletions reviewer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Intro to Reviewers build process

## Github Workflows

The Reviewers build automation uses two GH workflows located on below paths:

- [.github/workflows/reviewer-setup.yaml](/.github/workflows/reviewer-setup.yaml)
- [.github/workflows/reviewer-build-push.yaml](/.github/workflows/reviewer-build-push.yaml)

The first `setup` workflow gets triggered on each `pull_request` and `push` events and will gather a list of all exercises Reviewers that have its files changed and require a new build.

It will then trigger the second workflow `build-push` to:

- **on pull_requests**: build and push the Reviewer container with a tag `devopsacademyau/reviewer:<exercise_name>-<version>-pr-<pr_number>`
- **on push to master**: build and push the Reviewer container with tags `devopsacademyau/reviewer:<exercise_name>-<version>` and `devopsacademyau/reviewer:<exercise_name>-latest`


## Setup new exercises build

To add a new exercise to the build automation you will have to:

- add a new <exercise_name> folder under [/reviewer/exercises/](/reviewer/exercises)
- provide all the required files to build the reviewer within that folder (Dockerfile, scripts, etc)
- modify the [.github/workflows/reviewer-setup.yaml](/.github/workflows/reviewer-setup.yaml) adding the <exercise_name> as part of the `dorny/paths-filter@v2` action filters as per below:

```yaml
- uses: dorny/paths-filter@v2
id: changes
with:
list-files: json
# better to list individually to have a cleaner integration with matrix strategy when calling build jobs
filters: |
git03:
- 'reviewer/exercises/git03/**/*'
git04:
- 'reviewer/exercises/git04/**/*'
<exercise_name>:
- 'reviewer/exercises/<exercise_name>/**/*'
```

Once this is done, next time you submit a PR with changes within `'reviewer/exercises/<exercise_name>/**/*'` folder, the container will be build and pushed to DockerHub.
6 changes: 6 additions & 0 deletions reviewer/exercises/git03/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM debian:bullseye-slim
WORKDIR /app

ADD review.sh /app

CMD [ "bash", "review.sh"]
3 changes: 3 additions & 0 deletions reviewer/exercises/git03/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"version": "v1.0.0"
}
Empty file.
9 changes: 9 additions & 0 deletions reviewer/exercises/git03/review.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh
secret_file="my_env.txt"
if [[ -f "/exercise/$secret_file" ]];then
echo "::error title=Exercise submission failed::Secret file $secret_file still exists. Please update your code and submit again."
exit 1;
fi;

echo "SUMMARY: ### Exercise completed :rocket:"
echo "SUMMARY: Secret file $secret_file not found. Well done cleaning all your secrets."
9 changes: 9 additions & 0 deletions reviewer/exercises/git04/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM python:3.10-slim-bullseye
WORKDIR /app

ADD requirements.txt /app
ADD review.py /app

RUN pip3 install -r requirements.txt

CMD [ "python", "./review.py"]
3 changes: 3 additions & 0 deletions reviewer/exercises/git04/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"version": "v1.0.0"
}
Empty file.
21 changes: 21 additions & 0 deletions reviewer/exercises/git04/review.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import sys

def github_error(message):
print("::error title=Exercise submission failed::{}".format(message))

def github_summary(message):
print("SUMMARY: {}".format(message))


secret_file = "my_env.txt"

try:
my_env = open("/exercise/{}".format(secret_file))

except:
github_summary("### Exercise completed :rocket:")
github_summary("Secret file {} not found. Well done cleaning all your secrets.".format(secret_file))
sys.exit()

github_error("Secret file {} still exists. Please update your code and submit again.".format(secret_file))
raise Exception("Submission failed.")
26 changes: 26 additions & 0 deletions students/caiocezart/exercises/git04/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## Secrets (c01-git04)

Perform the following commands:
1. In the same repository of the previous exercise:
2. Add a file called `my_env.txt` that contains
```
SERVICE_NAME=account-management
ENVIRONMENT=prod
PASSWORD=pass1234
```
1. Commit it to your local repository
2. Check the log with `git log`

**Questions**

1. Let's suppose you remotely pushed the `my_env.txt` file above. A colleague asks you to remove this information from Git. What's your colleague worried about?
2. If you modify the file in your workspace, then commit and push it, will it be enough to erase this password information from the repository? (It's not). Why?
3. If you delete the file and push it, then create a new one with the rest of the information, is it enough? (It's not). Why?
4. How to fix this? How do you remove something from Git history when it is in the remote repository?
5. Which commands would you use? Explain what the command does.

## Submit a PR with the following files

> Remember to follow the instructions on [how to submit a PR here](/README.md#exercises)

- **README.md**: copy from file [ANSWER.md](ANSWER.md), answering the questions above. Include details and commands used.
3 changes: 3 additions & 0 deletions students/caiocezart/exercises/git04/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"version": "v1.0.0"
}
3 changes: 3 additions & 0 deletions students/caiocezart/exercises/git04/my_env.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
SERVICE_NAME=account-management
ENVIRONMENT=prod
PASSWORD=pass1234