Possible pre-training data leakage?

[wzhang2022]

I'm currently working on a project where I pre-train a neural network on ogbg-molpcba to improve performance on ogbg-molhiv. However, I've noticed that there seems to be some overlapping molecules in the training set of ogbg-molpcba and the test set ogbg-molhiv. If the goal is to demonstrate that a certain pre-training method improves performance, is it important to exclude these overlapping molecules in the pre-training? If so, did this paper also account for this?

[weihua916]

Yes, we have accounted for that: Section 5.1 of our pre-training paper "Furthermore, to prevent data
leakage, all test graphs used for performance evaluation are removed from the graph-level supervised
pre-training datasets."

For the purpose of the leaderboard, we only allow unsupervised pre-training to keep the comparison fair. In fact, some labels in ogbg-pcba may be highly correlated with that of ogbg-molhiv. For the purpose of your investigation, you should feel free to investigate supervised pre-training.