Skip to content

Commit

Permalink
Update 2024-03-01-policy.md
Browse files Browse the repository at this point in the history
  • Loading branch information
infwinston authored Jun 29, 2024
1 parent 17e93c6 commit 3839763
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion blog/2024-03-01-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ To ensure the leaderboard accurately reflects model rankings, we rely on live co

**Sharing data with the community**: We will periodically share data with the community. In particular, we will periodically share 20% of the arena vote data we have collected including the prompts, the answers, the identity of the model providing each answer (if the model is or has been on the leaderboard), and the votes. For the models we collected votes for but have never been on the leaderboard, we will still release data but we will label the model as "anonymous".

**Sharing data with the model providers**: Upon request, we will offer early data access with model providers who wish to improve their models. However, this data will be a subset of data that we periodically share with the community. In particular, with a model provider, we will share the data that includes their model's answers. For battles, we may not reveal the opponent model and may use "anonymous" label. This data will be later shared with the community during the periodic releases. If the model is not on the leaderboard at the time of sharing, the model’s answers will also be labeled as "anonymous".
**Sharing data with the model providers**: Upon request, we will offer early data access with model providers who wish to improve their models. However, this data will be a subset of data that we periodically share with the community. In particular, with a model provider, we will share the data that includes their model's answers. For battles, we may not reveal the opponent model and may use "anonymous" label. This data will be later shared with the community during the periodic releases. If the model is not on the leaderboard at the time of sharing, the model’s answers will also be labeled as "anonymous". Before sharing the data, we will remove user PII (e.g., Azure PII detection for texts).

## FAQ

Expand Down

0 comments on commit 3839763

Please sign in to comment.