Compability with higher version of keras

[grzegorzwojdyga]

Hi, I have tried to make your code compatible with higher version of keras - 2.2.24 (happy to do pull request later), I have changed only couple of lines in model/lang_model_sgd.py:

    def get_updates(self, params, loss):
        grads = self.get_gradients(loss, params)
        self.updates = []
        self.updates.append(K.update_add(self.iterations, 1))
        for p, g in zip(params, grads):
            new_ = p - self.lr * g
            if getattr(p, 'constraint', None) is not None:
                new_p = p.constraint(new_p)
            self.updates.append(K.update(p, new_p))
    return self.updates

and it worked. But the calculation of perplexity seems to be wrong. After first epoch I have:

Epoch 1/40
2232/2232 [==============================] - 292s 131ms/step - loss: 6.7872 - acc: 0.0613 - perplexity: 8299147.2073 - val_loss: 6.6075 - val_acc: 0.0683 - val_perplexity: 8149344.1497

Do you have any idea what might be causing it? I have seen that you had some issues with perplexity earlier so maybe you have instant answer for this matter. I'll be grateful for any feedback.