[PTFE-433] ✨ Add workflow cost metric

[gaspardmoindrot]

No description provided.

[codecov-commenter]

Codecov Report

Merging #44 (d0b786d) into main (6f4b71e) will increase coverage by 1.23%.
The diff coverage is 97.63%.

@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   96.15%   97.39%   +1.23%     
==========================================
  Files          10       14       +4     
  Lines         442      537      +95     
==========================================
+ Hits          425      523      +98     
+ Misses         17       14       -3     

Flag	Coverage Δ
api	97.31% <97.50%> (?)
unit	18.63% <32.72%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
gh_actions_exporter/main.py	96.15% <88.88%> (-1.41%)	⬇️
gh_actions_exporter/metrics.py	92.91% <93.75%> (+4.58%)	⬆️
tests/api/metrics/test_job.py	97.97% <97.97%> (ø)
gh_actions_exporter/Webhook.py	100.00% <100.00%> (ø)
gh_actions_exporter/config.py	97.22% <100.00%> (+0.44%)	⬆️
gh_actions_exporter/cost.py	100.00% <100.00%> (ø)
gh_actions_exporter/types.py	100.00% <100.00%> (ø)
tests/api/conftest.py	100.00% <100.00%> (ø)
tests/api/metrics/conftest.py	100.00% <100.00%> (ø)
tests/api/metrics/test_workflow.py	100.00% <100.00%> (ø)
... and 2 more

[nootal]

Is there a reason we run twice the same check?

[gaspardmoindrot]

We don't actually run it twice; it's just that we simulate having a workflow with 2 jobs!

[tcarmet]

I don't understand why this metric was added, as I fail to see the difference with github_actions_job_cost_count given that the label workflow_name is available on it as well.

Would you mind sharing the query you are trying to do?

[gaspardmoindrot]

I don't understand why this metric was added, as I fail to see the difference with github_actions_job_cost_count given that the label workflow_name is available on it as well.

Would you mind sharing the query you are trying to do?

Certainly, in essence, what I wanted to do with the graph was to display the average cost per workflow, something like this:

sum(increase(github_actions_workflow_cost_count_total{repository=~"$repository", runner_type=~"$runner_type", repository_visibility=~"$repository_visibility", cloud=~"$cloud", workflow_name=~"$workflow_name"}[$__interval])) by (repository, workflow_name) / sum(increase(github_actions_workflow_total_count_total{repository=~"$repository", runner_type=~"$runner_type", repository_visibility=~"$repository_visibility", cloud=~"$cloud", workflow_name=~"$workflow_name"}[$__interval])) by (repository, workflow_name) > 0

As you know, github_actions_workflow_cost_count_total does not exist, so I wanted to add it. Initially, I tried to work around this issue by changing the query and finding an alternative approach, but without success. The information available to us is the cost of each job, the number of jobs, and the number of workflows. Based on that, I don't see how to calculate the average cost of each workflow. If you have any ideas, I'm open to them, and we can simply close this pull request.

If we had the number of different jobs within a specific workflow, we could make the calculation, but we don't have that value either.

[tcarmet]

Given how you are using this new metrics, you will reach the same result with the existing one:

sum(increase(github_actions_job_cost_count_total{repository=~"$repository", runner_type=~"$runner_type", repository_visibility=~"$repository_visibility", cloud=~"$cloud", workflow_name=~"$workflow_name"}[$__interval])) by (repository, workflow_name) / sum(increase(github_actions_workflow_total_count_total{repository=~"$repository", runner_type=~"$runner_type", repository_visibility=~"$repository_visibility", cloud=~"$cloud", workflow_name=~"$workflow_name"}[$__interval])) by (repository, workflow_name) > 0

Checkout how a graph can look like with it: https://mon.scality.net/grafana/d/WidbLgPnk/github-actions-monitoring?orgId=1&refresh=30s&viewPanel=72