Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in config_parse.py when batch_norm layer is used in RecurrentLayerGroup #966

Merged
merged 5 commits into from
Dec 21, 2016

Conversation

qingqing01
Copy link
Contributor

@qingqing01 qingqing01 commented Dec 20, 2016

Fix #961

解决方法:

在 config_parse.py里构造 BatchNormLayer的另外两个Input()时,不经过函数 https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer/config_parser.py#L503

错误原因:

产生原因是 BatchNorm Layer里自动为moving average参数添加了Input():

        for i in xrange(2):
            inputs.append(
                Input(
                    inputs[0].input_layer_name,
                    initial_std=0.0,
                    initial_mean=0.0,
                    is_static=True,
                    is_shared=is_shared,  ))

因为调用顺序原因,这里引用的名字 inputs[0].input_layer_name, 在RecurrentLayerGroup里是被函数MakeLayerNameInSubmodel()编码之后的名字,即 input_layer_name + "@" + submodel_name, 导致出错。

其他解决办法:

如果在config_parse.py外面, 由用户自定添加,例如下面方式,这样两个Input(1)和Input(2)的name经过config_parse.py内部编码会与 Input(0) 保持一致, 不会出错。

    Layer(name=name + 'batch_norm1',
          type='batch_norm',
          batch_norm_type=bn_type,
          active_type="relu",
          use_global_stats=use_global_stats,
          bias=Bias(initial_mean=0.0,
                    initial_std=0.0,
                    is_static=static,
                    learning_rate=lr),
          inputs=[Input(name + 'conv1',
                                initial_mean=0,
                                initial_std=0.1,
                                is_static=static,
                                learning_rate=lr,
                                image=Image(channels=size,
                                        img_size=1)),
                       Input(name + 'conv1',
                                initial_std=0.0,
                                initial_mean=0.0,
                                is_static=True, ),
                       Input(name + 'conv1',
                                initial_std=0.0,
                                initial_mean=0.0,
                                is_static=True, ),  ] )

这个可以在 trainer_config_helpers 里重新包装, 但是部分还是用老配置的用户,得手动自己添加,较麻烦。

其他办法:

@@ -498,9 +498,12 @@ def __init__(
is_static=None,
is_shared=None,
update_hooks=None,
input_layer_argument=None, ):
input_layer_argument=None,
not_make_layer_name_in_submodel=None, ):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请问这个变量到底之bool型的还是一个指针呀?

python经常把默认参数设置成None是有原因的,参考 http://effbot.org/zone/default-values.htm

对于简单类型,比如整数,bool型,那么默认值直接传就可以了。因为Python对于简单类型是值传递而不是引用传递。

Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qingqing01 qingqing01 merged commit bbf3b47 into PaddlePaddle:develop Dec 21, 2016
@qingqing01 qingqing01 deleted the batch_norm branch April 20, 2017 10:34
wangxicoding pushed a commit to wangxicoding/Paddle that referenced this pull request Dec 9, 2021
* Add offset mapping doc

* fix eval hang because of unique endpoint

* generate api support encoder-decoder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants