Skip to content
This repository has been archived by the owner on Jul 6, 2023. It is now read-only.

kernel_initializer="glorot_normal" #27

Open
puhan123 opened this issue Apr 20, 2022 · 4 comments
Open

kernel_initializer="glorot_normal" #27

puhan123 opened this issue Apr 20, 2022 · 4 comments

Comments

@puhan123
Copy link

In birealnet.py, I observe that kernel_initializer is set as "glorot_normal" for Conv2d; However, in QuantConv2D, the kernel_initializer is set as "glorot_normal".
Kernel_initializer ="glorot_normal" means the kernel weights are all set as 1 ?

Thank you very much.

@lgeiger
Copy link
Member

lgeiger commented Apr 20, 2022

Glorot normal draws samples from a truncated normal distribution centered on 0, so in this case it means binary weights will be randomly set to either -1 or 1.

@puhan123
Copy link
Author

Thank you very much. I still have one more doubt.
Are both Conv2d and QuantConv2D all set to either -1 or 1?

@lgeiger
Copy link
Member

lgeiger commented Apr 28, 2022

Are both Conv2d and QuantConv2D all set to either -1 or 1?

Only weights of QuantConv2D layers will be binarized.

@puhan123
Copy link
Author

puhan123 commented Apr 29, 2022

My problem is that I found that the weights of QuantConv2D obtained with initialization method "glorot_normal" is not +1 or -1.
"glorot_normal" in Pytorch framework is as below:
image

The example is as :
image

The value of w is apparently not +1 or -1.
Thank you very much.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants