Implement cbuild-pack.yml support

[Torbjorn-Svensson]

Fixes #1122

Contributed by STMicroelectronics

[github-actions]

Test Results

    7 files    53 suites   4m 9s ⏱️
188 tests 171 ✔️ 17 💤 0 ❌
700 runs  632 ✔️ 68 💤 0 ❌

Results for commit f912544.

♻️ This comment has been updated with latest results.

[brondani]

@Torbjorn-Svensson @ErikMallbergSTM Can you please explain the algorithm being implemented here? I see some pack resolution stuff that I wouldn't expect. My expectation would be to store in cbuild-pack.yml the information about processed packs similarly to what is already collected and stored in cbuild.yml files.
I don't care about internal implementation specific names, but externally in yml files and schemas the naming convention must be consistent, if possible please do not mix terms such as cbuild-pack and lockfile.

[Torbjorn-Svensson]

@Torbjorn-Svensson @ErikMallbergSTM Can you please explain the algorithm being implemented here? I see some pack resolution stuff that I wouldn't expect. My expectation would be to store in cbuild-pack.yml the information about processed packs similarly to what is already collected and stored in cbuild.yml files. I don't care about internal implementation specific names, but externally in yml files and schemas the naming convention must be consistent, if possible please do not mix terms such as cbuild-pack and lockfile.

Thanks for your input.
We have tried to address your comments in the last update.
Documentation of the algorithm is on the way and will be shared soon.

[Torbjorn-Svensson]

We are working on improving the documentation and addressing the remaining comments pr and in Open-CMSIS-Pack/cmsis-toolbox#76.

[Torbjorn-Svensson]

Hi, @soumeh01 were you suppose to do merges into this branch or some other branch? Otherwise, we will just rebase when we do the next push since there are still a few changes left.

[soumeh01]

Hi, @soumeh01 were you suppose to do merges into this branch or some other branch? Otherwise, we will just rebase when we do the next push since there are still a few changes left.

Ok, I was not aware there were still changes left and had updated the branch.
No worries we can wait for the leftover changes and then you can rebase it with main.

[Torbjorn-Svensson]

Some tests remain to be created to verify usage of context selection.

[Torbjorn-Svensson]

@jkrech @brondani This PR is now updated with the additional test. Please review it and see if there is anything left before this PR can be merged.

[Torbjorn-Svensson]

@jkrech Can you fix the coverage job that failed for some reason?

[jkrech]

I triggered the coverage action again and it completed successfully now.

[Torbjorn-Svensson]

@jkrech Looks like the coverage failed again. Can someone at Arm look into why this job appears to fail repeatedly?
For reference: https://github.com/Open-CMSIS-Pack/devtools/actions/runs/6942633876/job/18885984864?pr=1143

[soumeh01]

@jkrech Looks like the coverage failed again. Can someone at Arm look into why this job appears to fail repeatedly? For reference: https://github.com/Open-CMSIS-Pack/devtools/actions/runs/6942633876/job/18885984864?pr=1143

@Torbjorn-Svensson, it appears there's a Codecov server outage. Despite 3 attempts to upload coverage data, the job has failed. We'll need to wait for all the jobs to complete before I can retrigger the coverage job.
Issue: codecov/codecov-action#598

[Torbjorn-Svensson]

@jkrech Looks like the coverage failed again. Can someone at Arm look into why this job appears to fail repeatedly? For reference: https://github.com/Open-CMSIS-Pack/devtools/actions/runs/6942633876/job/18885984864?pr=1143

@Torbjorn-Svensson, it appears there's a Codecov server outage. Despite 3 attempts to upload coverage data, the job has failed. We'll need to wait for all the jobs to complete before I can retrigger the coverage job. Issue: codecov/codecov-action#598

As the same problem have happened a few times in a row now, would it be possible to give me the power to be able to retrigger the failed jobs?

[jkrech]

@Torbjorn-Svensson can you try to see whether you can see the re-run action now?

[codecov]

Codecov Report

Merging #1143 (f912544) into main (5575cbc) will increase coverage by 0.37%.
The diff coverage is 97.58%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1143      +/-   ##
==========================================
+ Coverage   58.78%   59.15%   +0.37%     
==========================================
  Files         117      117              
  Lines       23301    23528     +227     
  Branches    12947    13090     +143     
==========================================
+ Hits        13697    13918     +221     
- Misses       7345     7346       +1     
- Partials     2259     2264       +5     

Flag	Coverage Δ
buildmgr-cov	77.92% <ø> (ø)
packchk-cov	64.16% <ø> (ø)
projmgr-cov	84.24% <97.58%> (+0.56%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
tools/projmgr/src/ProjMgr.cpp	76.10% <100.00%> (+0.49%)	⬆️
tools/projmgr/src/ProjMgrYamlEmitter.cpp	95.31% <100.00%> (+0.76%)	⬆️
tools/projmgr/src/ProjMgrYamlSchemaChecker.cpp	88.46% <100.00%> (+0.46%)	⬆️
tools/projmgr/src/ProjMgrUtils.cpp	88.36% <96.87%> (+1.36%)	⬆️
tools/projmgr/src/ProjMgrWorker.cpp	84.46% <97.64%> (+0.11%)	⬆️
tools/projmgr/src/ProjMgrYamlParser.cpp	77.03% <92.68%> (+1.43%)	⬆️

[Torbjorn-Svensson]

@Torbjorn-Svensson can you try to see whether you can see the re-run action now?

I think I have what's needed now. I'll verify next time I see a broken build if I can retrigger.

[brondani]

LGTM.
@Torbjorn-Svensson Many thanks for this contribution.

For the posterity: in a future refactoring please considering unifying as much as possible the pack resolution for cbuild-pack handling and for pack loading (see remarks).