Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding new rocm triton flash attention kernel #4

Closed
wants to merge 8 commits into from
Closed

Adding new rocm triton flash attention kernel #4

wants to merge 8 commits into from

Commits on Mar 15, 2024

  1. Adding new rocm triton flash attention kernel 

    Co-authored-by: Vinayak Gokhale <[email protected]>
    @jpvillam-amd @vgokhale
    jpvillam-amd and vgokhale committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    e569133 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. Small fix on dockerfile

    @jpvillam-amd
    jpvillam-amd committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    0e63661 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into jpvillam/v0.3.3_triton

    @jpvillam-amd
    jpvillam-amd committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    c89c0e3 View commit details
    Browse the repository at this point in the history

  3. Rebase updates and PR review changes 

    Added Flag for controlling triton vs default flow.
More small changes to dockerfile
    @jpvillam-amd
    jpvillam-amd committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    d4cb905 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Added interleaving for MQA for triton kernel

    jpvillam committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    1fff99a View commit details
    Browse the repository at this point in the history

  2. Only if VLLM_USE_FLASH_ATTN_TRITON is to 1 or true

    @jpvillam-amd
    jpvillam-amd committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    a3bc6c8 View commit details
    Browse the repository at this point in the history

  3. Make triton the default FA

    @jpvillam-amd
    jpvillam-amd committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    8cd9653 View commit details
    Browse the repository at this point in the history

  4. Make workaround only applicable to triton path

    @jpvillam-amd
    jpvillam-amd committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    52af554 View commit details
    Browse the repository at this point in the history