About training loss

[L3LeTrigger-F]

Hi,we are training RQ-VAE with the code in this link: https://github.com/RUCAIBox/LC-Rec/tree/main/index
Why did total loss go down first and then up, but reconstruction loss went down gradually?
We ran experiments on our own data set, this is the trend of loss:

We also repeated the experiment on the original data set and found that the change trend of loss is the same. So what is the reason?

[L3LeTrigger-F]

ok..I solved the problem by setting the learning rate to 1e-2...

[L3LeTrigger-F]

new question. When the learning rate is adjusted to 1e-2, the collision rate is 0.99.

[wzyustc]

新问题。当学习率调整为1e-2时，碰撞率0.99。

您好，请问您的碰撞率是如何下降的呢，我训练时碰撞率一直维持在60%-70%。

[zhengbw0324]

@L3LeTrigger-F
We also found that the loss rose and fell periodically in the experiment, so we tried to use learning rate schedule. This will make the learning rate smaller in the final stage of training to achieve better convergence.

[zhengbw0324]

@wzyustc
您好，您的数据的item数量大概是多少呢？如果数量比较大可以尝试：
（1）将4级codebook改为5级，也可以将每级codebook从256个调大。
（2）调整encoder decoder结构，我们实验中发现更深层的encoder有利于降低冲突率。
总的来说，RQ-VAE在建模大量物品时还是存在很多局限性有待解决。