Skip to content

Latest commit

 

History

History
70 lines (55 loc) · 4.52 KB

use-continuous-integration.md

File metadata and controls

70 lines (55 loc) · 4.52 KB
Raw
order redirect_from
12
criteria/continuous-integration

Use continuous integration

Asynchronous collaboration is enabled by developers merging their work to a shared branch frequently, verified by automated tests. The more frequent the merging and the smaller the contribution, the easier it is to resolve merge conflicts.

Automated testing of all functionality provides confidence that contributions are working as intended and have not introduced errors, and allows reviewers to focus on the structure and approach of the contribution. The more focused the test, the easier it is to clearly identify and understand errors as they arise.

Documenting a codebase's continuous integration workflow helps contributors understand the expectations of contributions. Continuous integration allows for an easier monitoring of the state of the codebase.

Requirements

  • All functionality in the source code MUST have automated tests.
  • Contributions MUST pass all automated tests before they are admitted into the codebase.
  • The codebase MUST have guidelines explaining how to structure contributions.
  • The codebase MUST have active contributors who can review contributions.
  • Automated test results for contributions SHOULD be public.
  • The codebase guidelines SHOULD state that each contribution should focus on a single issue.
  • Source code test and documentation coverage SHOULD be monitored.
  • Testing policy and documentation for consistency with the source and vice versa is OPTIONAL.
  • Testing policy and documentation for style and broken links is OPTIONAL.
  • Testing the software by using examples in the documentation is OPTIONAL.

How to test

  • Confirm that there are tests present.
  • Confirm that source code coverage tools check that coverage is at 100% of the source code.
  • Confirm that contributions are only admitted into the codebase after all of the tests are passed.
  • Confirm that contribution guidelines explain how to structure contributions.
  • Confirm that there are contributions from within the last three months.
  • Check that test results are viewable.
  • Check if source code coverage data is published.

Public policy makers: what you need to do

  • Involve managers as well as developers and designers as early in the process as possible and keep them engaged throughout development of your policy.
  • Make sure there are also automated tests set up for policy documentation.
  • Fix policy documentation promptly if it fails a test.
  • Make sure the source code reflects any changes to the policy (see Maintain version control).

Managers: what you need to do

  • Make sure to test with real end users as quickly and often as possible.
  • Plan the work to integrate small parts very often instead of large parts less frequently.
  • Procure consultancy services that deliver incrementally aligned with the plan.
  • After a large failure, encourage publication of incident reports and public discussion of what was learned.

Developers and designers: what you need to do

  • Help managers structure the work plan such that it can be integrated as small increments.
  • Help contributors limit the scope of their contributions and feature requests to be as small as reasonable.
  • Help managers and policy makers test their contributions, for example by testing their contributions for broken links or style.
  • Structure source code written to handle conditions which are difficult to create in a test environment in such a way that the conditions can be simulated during testing. Forms of resource exhaustion such as running out of storage space and memory allocation failure are typical examples of difficult to create conditions.
  • Tune the test code coverage tools to avoid false alarms resulting from inlining or other optimizations.
  • Deploy often.
  • Integrate your work at least once a day.

Further reading