Random initialization is better?

[zhiweihu1103]

Problem description: I initialize the parameters of NewLook randomly (that is, set the --init or --init_checkpoint parameter to None in run_model_NewLook.py), and find that the effect of using random initialization directly is better than the effect presented in the paper (take FB15k-237 as example). Can you explain it?
Below is the results table:

Notes:
NLK refers to the results of your paper;
VAL refers to the valid set results after training 30000 steps, and use the saved model to predict.
TST refers to the test set results with the random initialization parameters for NewLook model.
We can see, the effect of TST is often better, Very confused about this result. Looking forward to your reply

[lihuiliullh]

what is the command you use for random initialization?

[zhiweihu1103]

what is the command you use for random initialization?

I modify the run_model_NewLook.py only removed the train related code, and set the --init parameter to None for random initialization. Just only do this.

[lihuiliullh]

Thanks for pointing the problem out. I just fixed the problem. It should work correctly now.

[zhiweihu1103]

Thanks for pointing the problem out. I just fixed the problem. It should work correctly now.

Thank you for your quickly fix. Except above, I have other questions:
Q1: Would you mind share the code about how you create the difference datasets (such as the test_triples_3d.pkl and so on with the d flag);
Q2: There are also some doubts about the generation of test and valid hard version data sets. Whether it is convenient for you to open source the code for generating hard version data(such as the test_ans_ci_hard.pkl)?
Looking forward to your reply. Thx.