Skip to content

Commit

Permalink
Guilherme/proto 3082 fix actor score ema initialization (#714)
Browse files Browse the repository at this point in the history
## Purpose of Changes and their Description

This PR introduces a new score initialization mechanism for new
participants using the formula:
`initialEmaScore = lowestEmaScoreInActiveSet - lambda *
standardDeviationOfEmaScoresInActiveSet`

This provides a reasonable starting point that requires consistent good
performance to enter the active set.

The lambda parameter (lambda > 0) allows governance to adjust how
challenging it is for new participants to enter the active set:
- Higher lambda: More conservative, requiring longer consistent
performance
- Lower lambda: More lenient, allowing faster entry into the active set

We update the topic's initial EMA score for each actor (inferer,
forecaster, and reputer) during reward distribution.

When an actor submits their payload, we check if they are a new actor.
If they are, we initialize their score using the topic's initial EMA
score, calculated using the active set's statistics.

## Link(s) to Ticket(s) or Issue(s) resolved by this PR

https://linear.app/alloralabs/issue/PROTO-3082/fix-actor-score-ema-initialization

## Are these changes tested and documented?

- [ ] If tested, please describe how. If not, why tests are not needed.
- [ ] If documented, please describe where. If not, describe why docs
are not needed.
- [ ] Added to `Unreleased` section of `CHANGELOG.md`?

## Still Left Todo

*Fill this out if this is a Draft PR so others can help.*
  • Loading branch information
kpeluso authored Dec 19, 2024
2 parents 274c20e + e9d838b commit 250da2b
Show file tree
Hide file tree
Showing 27 changed files with 3,466 additions and 1,875 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* [#694](https://github.com/allora-network/allora-chain/pull/694) Make fuzzer whitelist aware
* [#708](https://github.com/allora-network/allora-chain/pull/708) Add Emission Enabled bolean flag to Mint Module
* [#703](https://github.com/allora-network/allora-chain/pull/703) Add outlier detection to inferences
* [#714](https://github.com/allora-network/allora-chain/pull/714) Add initialization of actors' EMA scores

### Changed

Expand Down
3 changes: 3 additions & 0 deletions test/integration/update_params_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ func UpdateParamsChecks(m testCommon.TestConfig) {
MinExperiencedWorkerRegrets: nil,
InferenceOutlierDetectionThreshold: nil,
InferenceOutlierDetectionAlpha: nil,
LambdaInitialScore: nil,
},
}
txResp, err := m.Client.BroadcastTx(ctx, m.AliceAcc, updateParamRequest)
Expand Down Expand Up @@ -152,6 +153,7 @@ func UpdateParamsChecks(m testCommon.TestConfig) {
MinExperiencedWorkerRegrets: nil,
InferenceOutlierDetectionThreshold: nil,
InferenceOutlierDetectionAlpha: nil,
LambdaInitialScore: nil,
},
}
_, err = m.Client.BroadcastTx(ctx, m.BobAcc, updateParamRequest)
Expand Down Expand Up @@ -217,6 +219,7 @@ func UpdateParamsChecks(m testCommon.TestConfig) {
MinExperiencedWorkerRegrets: nil,
InferenceOutlierDetectionThreshold: nil,
InferenceOutlierDetectionAlpha: nil,
LambdaInitialScore: nil,
},
}
txResp, err = m.Client.BroadcastTx(ctx, m.AliceAcc, updateParamRequest)
Expand Down
Loading

0 comments on commit 250da2b

Please sign in to comment.