[repo] CI improvements #5023

CodeBlanch · 2023-11-03T16:24:12Z

Changes

We recently overhauled the CI on the contrib repo to support more granular selection of jobs/tasks so that we can run subsets of the pipeline based on what is being changed. This PR brings that set up over to the main repo. We no longer need the "-md" versions of workflows and we can now be more selective about what we run.

The main workflow "ci.yml" now decides what to run based on the files being changed. High-level overview:
- We run the sanity check job for all changes.
- If a ".md" file is changed we run the markdown lint job.
- If some code artifact is changed (".cs", ".csproj", ".editorconfig") we run the dotnet format lint job.
- If code is changed or something in the build folder is changed we run a build+test of the solution in stable & experimental modes.
- If code is changed for an instrumentation package we run a build+test of the InstrumentationLibraries.proj in stable & experimental modes against the stable NuGet versions of dependencies.
- If OTLP exporter code is changed (or one of its dependencies) we run the OTLP integration tests.
- If api, sdk, or instrumentation is changed we run the W3C integration tests. This is too broad IMO and needs further work.
- If any packaged code is changed we run the package validation workflow.
- If any packaged code is changed we run the docfx workflow.
- If any packaged code is changed we run the AOT verification workflow.
All build+test runs have been refactored into a reusable workflow Component.BuildTest.yml. The main goal with that is to reduce the amount duplicated code and spots that need to be updated when we add a new TFM. All the steps also now have unique names so the checks UI is easier to read. After this PR the only required action will be "build-test" which is the step in the main workflow which joins everything together once complete.

This reverts commit 9e51290.

codecov · 2023-11-03T16:30:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6250307) 83.38% compared to head (c8a78fc) 83.18%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5023      +/-   ##
==========================================
- Coverage   83.38%   83.18%   -0.21%     
==========================================
  Files         297      271      -26     
  Lines       12531    11968     -563     
==========================================
- Hits        10449     9955     -494     
+ Misses       2082     2013      -69

Flag	Coverage Δ
unittests	`?`
unittests-Instrumentation-Experimental	`25.05% <ø> (?)`
unittests-Instrumentation-Stable	`25.05% <ø> (?)`
unittests-Solution-Experimental	`83.10% <ø> (?)`
unittests-Solution-Stable	`83.16% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 32 files with indirect coverage changes

github-actions · 2023-11-11T03:17:00Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

utpilla · 2023-11-17T19:12:42Z

@CodeBlanch Could you also do this for the Coyote checks introduced in #4879?

github-actions · 2023-11-28T03:16:49Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

TimothyMothra · 2023-11-30T17:17:22Z

Love this!
If I could add one thing, I'd like to see a brief comment at the top of workflows that describes ci.yml as the entry point that invokes all others. I also like to include one link at the top of workflows that best describes how they work. Seems this is the most appropriate: https://docs.github.com/en/actions/using-workflows/reusing-workflows#creating-a-reusable-workflow

…elemetry-dotnet into ci-improvements

.github/workflows/ci.yml

CodeBlanch · 2023-12-08T17:42:38Z

@TimothyMothra I pushed an update adding some comments into the child workflows.

cijothomas · 2023-12-15T18:56:38Z

.github/workflows/ci.yml

+          packaged-code: ['src/**', '!**/*.md']
+          api-code: ['*/OpenTelemetry.Api*/**', '!**/*.md']
+          api-packages: ['src/OpenTelemetry.Api*/**', '!**/*.md']
+          instrumentation: ['*/OpenTelemetry.Instrumentation*/**', 'test/TestApp.AspNetCore/**', '!**/*.md']


given this repo only has well defined components this is okay. The contrib repo needs some changes to the wording in https://github.com/open-telemetry/opentelemetry-dotnet-contrib/blob/main/CONTRIBUTING.md#contributing-a-new-project I believe, so that new components get CI covered. Not a blocker for this PR, just something to be addressed in contrib repo.

Any new component added in the contrib repo would still get CI covered. It's just that if the new component does not a component specific proj file, any changes to the component would trigger CI for every project in the solution that does not have a dedicated proj file. That's because the filtering logic would determine that the CI needs to run for every project that does not have a dedicated proj file.

We probably should update the CONTRIBUTING.md to mention that new contributios could have a more efficient CI check by introducing a dedicated proj file for their component.

utpilla · 2023-12-15T19:56:54Z

@CodeBlanch Need this change here as well? open-telemetry/opentelemetry-dotnet-contrib#1487

CodeBlanch · 2023-12-15T20:08:06Z

@utpilla

Need this change here as well? open-telemetry/opentelemetry-dotnet-contrib#1487

Good question! We don't need the powershell part because we aren't using that in this repo. The other bits from that PR...

This we already have in the repo:

opentelemetry-dotnet/build/Common.props

Line 50 in af83eb1

<MSBuildTreatWarningsAsErrors>true</MSBuildTreatWarningsAsErrors>
The bit to skip undefined TFMs when running tests is being added on this PR: https://github.com/open-telemetry/opentelemetry-dotnet/pull/5023/files#diff-5e21151922a3cc1af4e89f73a993199997928b4299905fcedc5a9de10d9294d5

utpilla · 2023-12-16T01:00:40Z

.github/codecov.yml

+  - "build"
+  - "docs"
+  - "examples"
+  - "src/Shared"


Should shared files under src be excluded?

Good question. This was copied from contrib but probably shouldn't be here. I'm thinking I'm going to leave it alone to see how codecov handles the new ci and come back to this later to smooth out any rough edges. I haven't figured out a way to make codecov really useful yet on contrib either 🤣

utpilla · 2023-12-16T01:03:48Z

.github/codecov.yml

+  unittests-Instrumentation-Stable:
+    carryforward: true
+    paths:
+      - src


Update path for instrumentation?

Not sure on this either. InstrumentationLibraries.proj actually builds and tests everything so this should be fine but 🤷 I'm considering code coverage as a WIP!

utpilla

This is a great improvement! Left some comments about codecov.yml which if need to be addressed could be addressed in another PR.

github-actions · 2023-12-30T03:16:50Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

github-actions · 2024-01-07T03:16:52Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

CodeBlanch added 19 commits October 20, 2023 13:26

WIP porting workflow improvements from contrib to main.

e19c474

CI cleanup.

0e02a27

Merge from main.

4efbae7

Fix.

6b4532a

Fixes.

5ddb458

Merge remote-tracking branch 'upstream/main' into ci-improvements

4300138

Remove non-ascii character.

600313e

Tweak.

79816af

Tweak.

31b789c

Fixes.

d776b6d

Fixes.

2db4dad

Skip target.

112ce8a

Tweaks and cleanup.

850f630

Merge from main.

5f26201

Manual merge of dotnet-format changes.

76d73ad

Tweak.

67f5bae

Revert net8.0 for w3c integration tests.

6f2305b

Test build without depth checkout.

9e51290

Revert "Test build without depth checkout."

75dfdbc

This reverts commit 9e51290.

CodeBlanch requested a review from a team November 3, 2023 16:24

github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Nov 11, 2023

Merge branch 'main' into ci-improvements

ed0d044

github-actions bot removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Nov 16, 2023

Merge branch 'main' into ci-improvements

be20741

Yun-Ting added 3 commits November 17, 2023 15:36

merge main

c37d22f

update concurrency-tests ci

7d55fb9

fixed location

4f5870b

github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Nov 28, 2023

Merge branch 'main' into ci-improvements

60768c5

github-actions bot removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Nov 29, 2023

CodeBlanch added 3 commits December 7, 2023 10:20

Merge from main.

951b2a7

Merge branch 'ci-improvements' of https://github.com/CodeBlanch/opent…

2f809cd

…elemetry-dotnet into ci-improvements

Bump actions/setup-dotnet from 3 to 4

e1c9d0b

utpilla reviewed Dec 8, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

utpilla reviewed Dec 8, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

CodeBlanch added 3 commits December 8, 2023 09:36

Merge from main.

e6ee01b

Add docker-compose.net8.0.yml back to the solution.

e1d00ad

Code review.

5b3418e

Merge remote-tracking branch 'upstream/main' into ci-improvements

5d700b9

cijothomas reviewed Dec 15, 2023

View reviewed changes

utpilla reviewed Dec 16, 2023

View reviewed changes

utpilla approved these changes Dec 16, 2023

View reviewed changes

Merge branch 'main' into ci-improvements

1549535

github-actions bot added Stale Issues and pull requests which have been flagged for closing due to inactivity and removed Stale Issues and pull requests which have been flagged for closing due to inactivity labels Dec 30, 2023

github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Jan 7, 2024

Merge branch 'main' into ci-improvements

c8a78fc

CodeBlanch merged commit 88493d2 into open-telemetry:main Jan 8, 2024
48 checks passed

CodeBlanch deleted the ci-improvements branch January 8, 2024 21:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[repo] CI improvements #5023

[repo] CI improvements #5023

CodeBlanch commented Nov 3, 2023 •

edited

Loading

codecov bot commented Nov 3, 2023 •

edited

Loading

github-actions bot commented Nov 11, 2023

utpilla commented Nov 17, 2023

github-actions bot commented Nov 28, 2023

TimothyMothra commented Nov 30, 2023

CodeBlanch commented Dec 8, 2023

cijothomas Dec 15, 2023

utpilla Dec 16, 2023 •

edited

Loading

utpilla Dec 16, 2023

utpilla commented Dec 15, 2023

CodeBlanch commented Dec 15, 2023

utpilla Dec 16, 2023

CodeBlanch Jan 8, 2024

utpilla Dec 16, 2023

CodeBlanch Jan 8, 2024

utpilla left a comment

github-actions bot commented Dec 30, 2023

github-actions bot commented Jan 7, 2024

[repo] CI improvements #5023

[repo] CI improvements #5023

Conversation

CodeBlanch commented Nov 3, 2023 • edited Loading

Changes

codecov bot commented Nov 3, 2023 • edited Loading

Codecov Report

github-actions bot commented Nov 11, 2023

utpilla commented Nov 17, 2023

github-actions bot commented Nov 28, 2023

TimothyMothra commented Nov 30, 2023

CodeBlanch commented Dec 8, 2023

cijothomas Dec 15, 2023

Choose a reason for hiding this comment

utpilla Dec 16, 2023 • edited Loading

Choose a reason for hiding this comment

utpilla Dec 16, 2023

Choose a reason for hiding this comment

utpilla commented Dec 15, 2023

CodeBlanch commented Dec 15, 2023

utpilla Dec 16, 2023

Choose a reason for hiding this comment

CodeBlanch Jan 8, 2024

Choose a reason for hiding this comment

utpilla Dec 16, 2023

Choose a reason for hiding this comment

CodeBlanch Jan 8, 2024

Choose a reason for hiding this comment

utpilla left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 30, 2023

github-actions bot commented Jan 7, 2024

CodeBlanch commented Nov 3, 2023 •

edited

Loading

codecov bot commented Nov 3, 2023 •

edited

Loading

utpilla Dec 16, 2023 •

edited

Loading