Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify two team names for single account #108

Open
kocmitom opened this issue Apr 13, 2023 · 3 comments
Open

Unify two team names for single account #108

kocmitom opened this issue Apr 13, 2023 · 3 comments
Milestone

Comments

@kocmitom
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
I am not sure how it happens, but there can be different names in the Ocelot leaderboard and different name then on files we use for further evaluation. Could these two team names be unified?

@kocmitom kocmitom added the wmt23 label Apr 13, 2023
@cfedermann
Copy link
Collaborator

cfedermann commented Apr 13, 2023 via email

@kocmitom
Copy link
Collaborator Author

Perhaps this:
"institution_name": "Tartu NLP",
"name": "Tartu NLP",
"publication_name": "Liv4ever",

But I am not sure if it is the source of the issue. Usually, those have been 1:1 but some teams decided to name their team differently (as in the exampel) than their systems and then it was detective work to figure out which system is whose (without access to Ocelot).
The easiest would be to drop redundant names and just keep team name = system name. What you think?

@cfedermann
Copy link
Collaborator

There is some logic in _team_name() to generate the name based on the publication_name but fall back to using the actual file's original_name when the former is missing/fails. This may explain the differences you have observed.

That being said, at export time, the file names we use for the results archive should match the expected publication_name value. Let's look into this once we have a first couple of submissions, so that we can define exactly what we have to fix/change.

@cfedermann cfedermann added this to the WMT23 milestone Jul 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants