Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Issue]: Forward kernel returns NaN when inputs are irregular (including causal) and sm_scale is 0 #47

Open
xinyazhang opened this issue Oct 7, 2024 · 1 comment
Open

[Issue]: Forward kernel returns NaN when inputs are irregular (including causal) and sm_scale is 0 #47

xinyazhang opened this issue Oct 7, 2024 · 1 comment

Comments

@xinyazhang
Copy link
Collaborator

Problem Description

If input shapes are irregular, qk will contain -inf, and -inf * 0.0 = nan

Consider specializing against sm_scale = 0

Operating System

Any

CPU

Any

GPU

AMD Instinct MI300X

ROCm Version

ROCm 6.2.0

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response
@xinyazhang xinyazhang changed the title [Issue]: Forward kernel returns NaN when sm_scale is 0 [Issue]: Forward kernel returns NaN when inputs are irregular (including causal) and sm_scale is 0 Oct 7, 2024
xinyazhang added a commit that referenced this issue Oct 7, 2024
@xinyazhang
Convert NaN created by 0.0 (from sm_scale) * -inf (from masking) to zero
fb8cb14 
This fixes #47
@xinyazhang
Copy link
Collaborator Author

Will be fixed by fb8cb14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@xinyazhang