Are the results' variance large?

[yxchng]

                          |        A-150       | A-847 | PC-59 | PC-459 | PAS-21 | PAS-20 |       COCO         |
                          |  PQ  | mAP  | mIoU | mIoU  | mIoU  |  mIoU  |  mIoU  |  mIoU  |  PQ  | mAP  | mIoU |
fc-clip large             | 26.8 | 16.8 | 34.1 | 14.8  | 58.4  |  18.2  |  81.8  |  95.4  | 54.4 | 44.6 | 63.7 |
fc-clip large (reproduce) | 25.3 | 16.2 | 32.8 | 14.2  | 56.7  |  17.5  |  82.7  |  95.5  | 56.5 | 47.6 | 65.0 |

I tried retraining the ConvNeXt-Large model and the performance is quite a bit lower than the published results. Are the results' variance large, such that I have to rerun it a few times?

[cornettoyu]

Hi,

You can refer to the training log here

We kept the best checkpoint in terms of ADE20K PQ metric, as we also note that the last checkpoint usually tends to "overfit" to the COCO dataset and shows worse generalization to other datasets.