New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug report: shape of pre_attn #3

Open

Elliot-QxZhang opened this issue Sep 16, 2024 · 0 comments

Elliot-QxZhang commented Sep 16, 2024 •

edited

Loading

Thank you for your great work. However, when I tried to train on TCGA-BRCA, I got an error:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (131072x1024 and 768x768)

After debugging, I believe the error is in the mmssl.py (line 28)

pre_params = {
                "input_dim": self.patch_embedding_dim,
                "hidden_dim": self.patch_embedding_dim,
}

I believe the input_dim should be set to 1024 instead of the same as hidden_dim 768. I reset it and it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment