result of post processing

[fyangneil]

i use the provided model and post processing code on davis16 dataset, the final result is 69.3 which is lower than 71.5 reported in paper. I am not sure if the result is correct.

[antonilo]

I think you might have an issue with image sizes. Are you sure you are running the post-processing on the final resolution (640x480)? Did you use our provided script to compute the metric or did you implement your own?

[fyangneil]

@antonilo I tried again and still got 69.3 on DAVIS

The output mask is 480x854 not 640x480. I did use your provided model and post-processing code and did not change anything.

[antonilo]

Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation script? If not, I might need more details to debug the problem (TensorFlow version used, for example)

[fyangneil]

I did use your provided evaluation script. but I do not think I get the exactly same detection results as yours, because based on your detection results and evaluation tool, I can get the number reported in paper.

…

________________________________ From: Antonio Loquercio <[email protected]> Sent: Monday, October 26, 2020 12:37 PM To: antonilo/unsupervised_detection <[email protected]> Cc: Fan Yang <[email protected]>; Author <[email protected]> Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11) Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation script<https://github.com/antonilo/unsupervised_detection/blob/master/test_generator.py>? If not, I might need more details to debug the problem (TensorFlow version used, for example) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#11 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALAUWTU27SKZGKU6RCI33GTSMWQULANCNFSM4SMHJSJA>.

[antonilo]

Might be that you used a sub-optimal checkpoint. How many epochs did you train?

[fyangneil]

I use the model you provided in github Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Antonio Loquercio <[email protected]> Sent: Tuesday, October 27, 2020 4:55:08 AM To: antonilo/unsupervised_detection <[email protected]> Cc: Fan Yang <[email protected]>; Author <[email protected]> Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11) Might be that you used a sub-optimal checkpoint. How many epochs did you train? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#11 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALAUWTSWATX3SPWDXJVNHPLSMXO2ZANCNFSM4SMHJSJA>.

[WeidiXie]

@fyangneil Hi, anything follows this ? have you reproduced the same number ? eg. 71 ?

[fyangneil]

no, I cannot reproduce the number reported in paper

…

________________________________ From: Weidi <[email protected]> Sent: Friday, February 5, 2021 5:49 PM To: antonilo/unsupervised_detection <[email protected]> Cc: Fan Yang <[email protected]>; Mention <[email protected]> Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11) @fyangneil<https://github.com/fyangneil> Hi, anything follows this ? have you reproduced the same number ? eg. 71 ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#11 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALAUWTR44TO3FIZCSZEZRYDS5RYX3ANCNFSM4SMHJSJA>.

[antonilo]

Sorry, but it is very difficult to help without details. I would say that the problem is in the way images are pre-processed (reshaping and central cutting). The changes might have affected the ckpts. What if you train a new ckpt (from scratch or resuming from the one available)? The predicted masks have been produced with the model available online.

[fyangneil]

But, I did not change anything. I simply used the model online and run the test script for DAVIS dataset, the result is lower than the number reported in paper.

…

________________________________ From: Antonio Loquercio <[email protected]> Sent: Saturday, February 6, 2021 1:43 PM To: antonilo/unsupervised_detection <[email protected]> Cc: Fan Yang <[email protected]>; Mention <[email protected]> Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11) Sorry, but it is very difficult to help without details. I would say that the problem is in the way images are pre-processed (reshaping and central cutting). The changes might have affected the ckpts. What if you train a new ckpt (from scratch or resuming from the one available)? The predicted masks have been produced with the model available online. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#11 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALAUWTSVMTQ64UBN5E7MKB3S5WEVTANCNFSM4SMHJSJA>.

[fyangneil]

@WeidiXie Have you reproduced the number reported in paper?

[fyangneil]

Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation script? If not, I might need more details to debug the problem (TensorFlow version used, for example)
I used the script. the tensorflow version is 13.1.

[WeidiXie]

I see, thanks, I was just wondering the step-wise breakdown results, but I'll test this myself when I got time.