Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The results do not match the description in Readme.md #197

Open
huayangcs opened this issue Apr 7, 2023 · 0 comments
Open

The results do not match the description in Readme.md #197

huayangcs opened this issue Apr 7, 2023 · 0 comments
Assignees

Comments

@huayangcs
Copy link

hello, thanks for this amazing benchmark work. However, when I reproduced Pgexplainer's results using the data you provided, the trained model you provided, and the pgexplainer model you provided, I found that the results were significantly different from the ones you provided in Readme.md. May I ask the possible reasons for this result? Thanks for your help.

DATASETS=ba_2motifs
python -m benchmarks.xgraph.pgexplainer_edges datasets=$DATASETS explainers=pgexplainer

results are :

Fidelity: 0.4920
Fidelity_inv: 0.4253
Sparsity: 0.5066
Accuracy: 0.04254115393283751
Stability: nan

after I change the config.explainers.sparsity = 0.8, the results are

Fidelity: 0.3911
Fidelity_inv: 0.4889
Sparsity: 0.8089
Accuracy: 0.04254115393283751
Stability: nan

@CM-BF CM-BF assigned CM-BF and Oceanusity and unassigned CM-BF Apr 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants