Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import human evaluation results #10

Open
snukky opened this issue Dec 11, 2020 · 1 comment
Open

Import human evaluation results #10

snukky opened this issue Dec 11, 2020 · 1 comment
Labels
enhancement New feature or request
Milestone
WMT23 Final

Comments

@snukky
Copy link
Contributor

snukky commented Dec 11, 2020

Add a management command for importing human rankings for a given test set. Human scores should be displayed as another metric/column in the table with results.

In general, we would like to be able to import results from any metric.
@snukky snukky added the enhancement New feature or request label Dec 11, 2020
@cfedermann cfedermann added this to the WMT22 milestone Jul 29, 2022
@cfedermann cfedermann added the wmt22 Planned for WMT22 label Jul 29, 2022
@cfedermann
Copy link
Collaborator

This is planned for the wmt22 release. Goal is to have human scores collected via Appraise in https://ocelot.mteval.org/ in time for the WMT22 conference.

@cfedermann cfedermann removed the wmt22 Planned for WMT22 label Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
WMT23 Final
Development

No branches or pull requests
2 participants
@cfedermann @snukky