don't save inputs/outputs buffer of FlashAttenFunc to reduce memory usage for inference mode#1383

Merged

tridao merged 1 commit intoDao-AILab:mainfrom XiaobingSuper:xiaobing/reduce_memory

Dec 12, 2024

+79-58

Commits on Dec 12, 2024

don't save inputs buffer of FlashAttenFunc to reduce memory usage for inference mode
XiaobingSuper
committed