Skip to content

Pull requests: Dao-AILab/flash-attention

New pull request New
52 Open 181 Closed
52 Open 181 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix bug when is_grad is false
#1406 opened Dec 22, 2024 by woaixiaoxiao Loading…
Add missing tests/__init__.py
#1405 opened Dec 20, 2024 by BioGeek Loading…
1
Fix incorrect torch dtype
#1399 opened Dec 19, 2024 by kevmo314 Loading…
check torch.is_grad_enabled before calling customer flash atten ops
#1397 opened Dec 19, 2024 by XiaobingSuper Loading…
1
Create PEP 517 build metadata
#1394 opened Dec 18, 2024 by frostming Loading…
Add hipBLAS/cuBLAS distinction in benchmark_gemm.py
#1393 opened Dec 17, 2024 by garrettbyrd Loading…
fix a bug (issue #1390) caused by typo
#1392 opened Dec 17, 2024 by liguohao96 Loading…
Support dedicated compile[For Research]
#1384 opened Dec 12, 2024 by AllenDou • Draft
Fix deprecation warnings
#1382 opened Dec 12, 2024 by rongou Loading…
wrap func into torch ops to avoid torch.compile graphbreaks
#1333 opened Nov 13, 2024 by kumarkrishna Loading…
Promote wheels as alternative to pip install flash-attn
#1297 opened Oct 25, 2024 by simonw Loading…
4
fix: in newer versions of triton, tl.dot should take as input only q …
#1288 opened Oct 21, 2024 by EdouardYvinec Loading…
flashattnvarlen support tree attention
#1188 opened Aug 30, 2024 by efsotr Loading…
the test_flash_attn.py it's actually in parent directory
#1167 opened Aug 21, 2024 by ArtificialZeng Loading…
Add support for qk hidden dim different from v hidden dim
#1166 opened Aug 20, 2024 by smallscientist1 Loading…
5
add softmax_d for mha_bwd
#1161 opened Aug 19, 2024 by MayDomine Loading…
Fix: bwd may need to first allocate cuda mem for rng_state
#1077 opened Jul 20, 2024 by jundaf2 Loading…
Windows actions
#1036 opened Jul 9, 2024 by bdashore3 Loading…
3
change condition to num_heads >= num_heads_k
#1030 opened Jul 5, 2024 by xenshinu Loading…
1
[Draft] support qk head_dim different from vo head_dim
#980 opened Jun 6, 2024 by defei-coder Loading…
1
2
Fix +/-inf in LSE returned by forward
#978 opened Jun 3, 2024 by sgrigory Loading…
1
add pyproject.toml with build dependencies
#958 opened May 17, 2024 by dhellmann Loading…
9
Relative position encoding
#956 opened May 14, 2024 by b-albar Loading…
1 of 4 tasks
1
ALiBi for the non-flash code path
#858 opened Feb 29, 2024 by Markus28 Loading…
Add local version identifier to package metadata for pre-built wheels
#856 opened Feb 28, 2024 by yundai424 Loading…
1
2
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.