-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add almanack score for simplified understanding through single metric #208
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Several discussion points this time
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
Co-authored-by: Gregory Way <[email protected]>
Co-Authored-By: Gregory Way <[email protected]>
… into sustainability-score
Thanks @gwaybio for the earlier review! I've restructured the approach here with an eye towards simplification and better documentation. When there's a chance, could I ask for another look? Thank you! |
Co-authored-by: Gregory Way <[email protected]>
Thanks @gwaybio for the further review! After making some changes based on your comments here and our discussion, I wanted to make note that I've changed the language a bit within the code and in the PR description to label this score as an "almanack score" which I feel might be distinct from the "sustainability score" (or at least may need consolidation at some point in the future to better encompass the full gamut of Almanack metrics). The PR no longer closes the sustainability score issue (#135) to instead continue exploring this as we develop the Almanack. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the great discussions in this PR! I think the project will benefit from them.
@@ -84,6 +83,16 @@ def get_table(repo_path: str) -> List[Dict[str, Any]]: | |||
for metric in metrics_table | |||
] | |||
|
|||
# calculate sustainability score (modify placeholder) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# calculate sustainability score (modify placeholder) | |
# calculate almanack score (the function modifies the placeholder) |
Computes an Almanack score by normalizing boolean Almanack | ||
table metrics in order to summarize analysis. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Computes an Almanack score by normalizing boolean Almanack | |
table metrics in order to summarize analysis. | |
Computes an Almanack score by counting boolean Almanack | |
table metrics to provide a quick summary of software sustainability. |
|
||
Returns: | ||
float: | ||
The computed almanack score, normalized between 0 and 1. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this still true?
Dictionary which includes number of almanack boolean metrics | ||
with a non-zero sustainability correction which passed, number | ||
of almanack checks in total, and a score which is the number | ||
of passed checks divided by the total number of checks that | ||
represents how likely a repository will be maintained over | ||
time based on the existing Almanack metrics. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dictionary which includes number of almanack boolean metrics | |
with a non-zero sustainability correction which passed, number | |
of almanack checks in total, and a score which is the number | |
of passed checks divided by the total number of checks that | |
represents how likely a repository will be maintained over | |
time based on the existing Almanack metrics. | |
Dictionary of length three, including the following: 1) number of | |
Almanack boolean metrics that passed (numerator), 2) number of | |
total Almanack boolean metrics considered (denominator), and 3) a | |
score that represents how likely the repository will be maintained | |
over time based (numerator / denominator). |
Description
This PR adds a "almanack score" metric which summarizes the findings of other metrics in one single value. This score is calculated using boolean metrics and a "sustainability direction" value per metric. The values are summed and divided by the number of metrics to then provide a score. This score is intended to prepare for a more statistically profound measure on repository sustainability which is formed through modeling and other related efforts.
Along the journey towards this PR I noticed and fixed the following:
References #135
Closes #207
What is the nature of your change?
Checklist
Please ensure that all boxes are checked before indicating that this pull request is ready for review.