Pre-training method differences between code and paper

[jakobamb]

Hi,

thank you for releasing your work.

In the paper you are writing that you use the SimMIM method with mean-image masks and spatial frequency masking. The same approach is described in the figure of this repository.

However, the code and description of this repo are describing pre-training based on BEiT, not SimMIM, and are not mentioning the frequency masking. Are you in the process of updating the repo?

Are the weights you published from BEiT pretraining, or from USFM pretraining described in MIA?

Thanks

Jakob

[BodongDu]

Hello,

I found the same question, the pre-training models in the code are Vit and SegVit. Where did you find that the model is BEiT. I find the Vit and SegVit in the task/Cls.yaml and task/Seg.yaml which are not consistent with paper. T am looking forward to your reply.

Best Reagrds,
Bodong Du

[BodongDu]

I have a little understanding that what we talked about is a specific architecture and the method talked in the paper is a self-supervised pre-training method which can be used in many architecture like BEiT or ViT

[jakobamb]

BEiT is also a SSL pre-training method, they all use ViTs as the model. Not sure where I read about BEiT, the code has been partially updated since then and now I cannot find any pretraining related code anymore

[BodongDu]

Well, I see.  We are looking forward to author's reply. In the new version, I can't find anything ahout pre-training.

…

---Original--- From: "Jakob ***@***.***> Date: Thu, Dec 5, 2024 18:35 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [openmedlab/USFM] Pre-training method differences between codeand paper (Issue #17) BEiT is also a SSL pre-training method, they all use ViTs as the model. Not sure where I read about BEiT, the code has been partially updated since then and now I cannot find any pretraining related code anymore — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>