Skip to content

Commit

Permalink
Update project root documentations reflecting Jenkins shutdown (apach…
Browse files Browse the repository at this point in the history
  • Loading branch information
Abacn authored Jan 26, 2024
1 parent 4981c0c commit 1d4413d
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 16 deletions.
37 changes: 24 additions & 13 deletions CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,27 +23,38 @@

Continuous Integration is important component of making Apache Beam robust and stable.

Our execution environment for CI is mainly the Jenkins which is available at
[https://ci-beam.apache.org/](https://ci-beam.apache.org/). See
[.test-infra/jenkins/README](.test-infra/jenkins/README.md)
for trigger phrase, status and link of all Jenkins jobs. See Apache Beam Developer Guide for
[Jenkins Tips](https://cwiki.apache.org/confluence/display/BEAM/Jenkins+Tips).

An additional execution environment for CI is [GitHub Actions](https://github.com/features/actions). GitHub Actions
(GA) are very well integrated with GitHub code and Workflow and it has evolved fast in 2019/2020 to become
a fully-fledged CI environment, easy to use and develop for, so we decided to use it for building python source
distribution and wheels.
Our execution environment for CI is the [GitHub Actions](https://github.com/features/actions).
See [.github/workflow/README](.github/workflow/README.md) for trigger phrase,
status and link of all GHA jobs.

GitHub Actions (GHA) are very well integrated with GitHub code and Workflow and
it has evolved fast in 2019/2020 to become a fully-fledged CI environment, easy
to use and develop for, so we decided to use it firstly for a few workflows then
migrated all workflows previously run on Jenkins.

For this reason there are mainly two types of GHA workflows running
- Self-hosted runner GHAs. These were mifrated from Jenkins with workflow name
prefix (beam_*.yml) as well as new workflows added following the same naming
convention, including PreCommit, PostCommit, LoadTest, PerformanceTest, and
several infrastructure jobs. See [.github/workflow/README](.github/workflow/README.md)
for the complete list.
- GitHub-hosted runner GHAs. Most of these jobs exercises the workflow in different
OS (linux, macOS, Windows). They were added prior to Jenkins migration.
Some Linux jobs later migrated to use self-hosted runner.

## GitHub Actions

This section applies to GitHub-hosted runner GHAs. New workflows unless intended
to run on a matrix of OS should refer to Self-hosted runner GHAs [.github/workflow/README](.github/workflow/README.md).

### GitHub actions run types

The following GA CI Job runs are currently run for Apache Beam, and each of the runs have different
purpose and context.

#### Pull request run

Those runs are results of PR from the forks made by contributors. Most builds for Apache Beam fall
Those runs are results of PR from the forks made by contributors. Most builds for Apache Beam fall
into this category. They are executed in the context of the "Fork", not main
Beam Code Repository which means that they have only "read" permission to all the GitHub resources
(container registry, code repository). This is necessary as the code in those PRs (including CI job
Expand All @@ -57,15 +68,15 @@ the PR is ready to review and merge.
Those runs are results of direct pushes done by the committers or as result of merge of a Pull Request
by the committers. Those runs execute in the context of the Apache Beam Code Repository and have also
write permission for GitHub resources (container registry, code repository).
The main purpose for the run is to check if the code after merge still holds all the assertions - like
The main purpose for the run is to check if the code after merge still holds all the assertions - like
whether it still builds, all tests are green.

This is needed because some of the conflicting changes from multiple PRs might cause build and test failures
after merge even if they do not fail in isolation.

#### Scheduled runs

Those runs are results of (nightly) triggered job - only for `master` branch. The
Those runs are results of (nightly) triggered job - only for `master` branch. The
main purpose of the job is to check if there was no impact of external dependency changes on the Apache
Beam code (for example transitive dependencies released that fail the build). Another reason for the nightly
build is that the builds tags most recent master with `nightly-master`.
Expand Down
3 changes: 0 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@
[![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[![Go tests (Jenkins)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon?subject=Go%20Tests%28Jenkins%29)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/)
[![Java tests (Jenkins)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon?subject=Java%20Tests%28Jenkins%29)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/)
[![Python tests (Jenkins)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Python%20Tests%28Jenkins%29)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)

## Overview

Expand Down

0 comments on commit 1d4413d

Please sign in to comment.