De-anonymise the primary systems for WMT21 news task

[bhaddow]

Hi

Now that we have a settled set of primary systems for WMT21, can we de-anonymise the primary systems (and only the primary systems)? The leaderboard showing all primary systems should be publicly visible,

best
Barry

[snukky]

Yes, we can de-anonymise them, but having a leaderboard showing all primary systems (not the top 10 submissions according to BLEU/ChrF) requires a little coding. We have already discussed internally how to implement that.

[cfedermann]

Reviving this thread: two follow-ups planned for WMT22:

1. Add support for leaderboard showing de-anonymised, full list of primary submissions for completed competitions; and
2. Merge old WMT21 data into the global, stable https://ocelot.mteval.org/ website. This one should remain online while the ocelot-wmt22.mteval.org instance should be considered unstable.

The goal is to get a stable OCELoT up and running while allowing individual year's to run their submission campaigns without any risks of data loss, plus with their own code base.

[cfedermann]

Maybe something to tackle after WMT23 submission week? It would be nice to get a stable ocelot.mteval.org version up and running...