New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kendall Tau in FAIRNAS #7

Open

pprp opened this issue Mar 8, 2021 · 1 comment

pprp commented Mar 8, 2021

Another paper EVALUATING THE SEARCH PHASE OF NEURAL ARCHITECTURE SEARCH tested FairNAS on NASBench101 but get the Kendall Tau of -0.23.

FairNAS using 13 models to evaulate the rank and get the Kendall Tau of 0.9487.

I think the number of models used in FairNAS is the way too little and can not really reflect the rank ability of FairNAS

cxxgtxy commented Aug 6, 2021

You can run the code in that paper and will find that the conclusion is not valid since the super-net is not trained at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment