Added Inverse Square Root Linear Unit (ISRLU) activation layer #456
Conversation
|
@RaphaelMeudec This is the new ISRLU PR. I have made all the changes you suggested when you reviewed the old one. |
There was a problem hiding this comment.
@SriRangaTarun Nice!
@gabrieldemarmiesse This feature can also be merged
Making keras-contrib compatible with tf.keras (keras-team#387)
|
@RaphaelMeudec and @gabrieldemarmiesse I have merged the tf.keras pull request into my branch and am trying to make ISRLU tf.keras compatible. The PSEU layer is now tf.keras compatible. |
|
@RaphaelMeudec and @gabrieldemarmiesse The build is now passing and the layer is tf.keras compatible. |
| self.trainable = False | ||
|
|
||
| if is_tf_keras: | ||
| def alpha_initializer(self, input_shape, dtype='float32', partition_info=None): |
There was a problem hiding this comment.
Same as the other PR, partition_info is not used and is_tf_keras if loop can be removed
There was a problem hiding this comment.
@RaphaelMeudec The initializer does not work in tf.keras without the partition_info argument.
There was a problem hiding this comment.
@gabrieldemarmiesse Added **kwargs
|
@fchollet It would be great if you or someone else from your team can review this PR. |
Please make sure you've read and understood our contributing guidelines;
https://github.com/keras-team/keras-contrib/blob/master/CONTRIBUTING.md
-->
- What I did
See above. Here's the axrXiv paper.
- How I did it
- How you can verify it
I added unit tests.
This pull request fixes #issue_number_here