Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect baseline job costs #132

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Conversation

cmelone
Copy link
Collaborator

@cmelone cmelone commented Nov 12, 2024

Pulled from #93 to collect data before deciding on final cost formula.

Adds the following columns:

  • nodes: AWS zone (zone)
  • nodes: Instance capacity type (capacity_type)
  • jobs: Cost of the instance during the lifetime of the job (job_cost_instance)

The job_cost_instance calculation is made by averaging the value of karpenter_cloudprovider_instance_type_offering_price_estimate during the lifetime of the node and multiplying by the duration of the build job.

This is not a cost per job metric. Use information like cpu_mean, mem_mean, etc to calculate the cost of the job in combination with job_cost_instance.

tested with dev/bulk_collect.py and verified that large migrations work correctly on the prod and staging db

Pulled from #93 to collect data before deciding on final cost formula.

Adds the following columns:
- nodes: AWS zone (`zone`)
- nodes: Instance capacity type (`capacity_type`)
- jobs:  Cost of the instance during the lifetime of the job (`job_cost_instance`)

The `job_cost_instance` calculation is made by averaging the value of `karpenter_cloudprovider_instance_type_offering_price_estimate` during the lifetime of the node and multiplying by the duration of the build job.

**This is not a cost per job metric.** Use information like cpu_mean, mem_mean, etc to calculate the cost of the job in combination with `job_cost_instance`.

tested with `dev/bulk_collect.py` and verified that large migrations work correctly on the prod and staging db
@cmelone cmelone requested a review from alecbcs November 12, 2024 22:53
@cmelone cmelone self-assigned this Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant