Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vllm multi lora rocm #25

Open
wants to merge 112 commits into
base: v0.4.0.post1-rocm
Choose a base branch
from
Open

Vllm multi lora rocm #25

wants to merge 112 commits into from

Commits on Oct 10, 2023

  1. Configuration menu
    Copy the full SHA
    43af310 View commit details
    Browse the repository at this point in the history
  2. port dtype_bfloat16.cuh

    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    cc81866 View commit details
    Browse the repository at this point in the history
  3. port attention_utils.cuh

    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    475b5e2 View commit details
    Browse the repository at this point in the history
  4. port more kernels

    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    ddc496c View commit details
    Browse the repository at this point in the history
  5. fix typo

    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    5eaa7a1 View commit details
    Browse the repository at this point in the history
  6. add cuda_compat.h

    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    f7273c6 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Configuration menu
    Copy the full SHA
    99c3be7 View commit details
    Browse the repository at this point in the history
  2. sync branches

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    f8093dc View commit details
    Browse the repository at this point in the history
  3. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    41df689 View commit details
    Browse the repository at this point in the history
  4. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    93be9c5 View commit details
    Browse the repository at this point in the history
  5. fixes

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    d96fa3c View commit details
    Browse the repository at this point in the history
  6. cleanup

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    421365b View commit details
    Browse the repository at this point in the history
  7. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    06b800e View commit details
    Browse the repository at this point in the history
  8. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    2312beb View commit details
    Browse the repository at this point in the history
  9. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    2958b39 View commit details
    Browse the repository at this point in the history
  10. fmt

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    3f89734 View commit details
    Browse the repository at this point in the history
  11. cleanup

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    5397a57 View commit details
    Browse the repository at this point in the history
  12. refactor

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    90e02d2 View commit details
    Browse the repository at this point in the history
  13. update

    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    a420202 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2023

  1. Configuration menu
    Copy the full SHA
    b072182 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Configuration menu
    Copy the full SHA
    2d1e435 View commit details
    Browse the repository at this point in the history
  2. using asm volatile instead of hip api

    iAmir97 authored and pcmoritz committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    e231b79 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    31bb335 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Configuration menu
    Copy the full SHA
    b027d06 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Add multi-LoRA support

    ---------
    
    Co-authored-by: Chen Shen <[email protected]>
    Co-authored-by: Shreyas Krishnaswamy <[email protected]>
    Co-authored-by: Avnish Narayan <[email protected]>
    4 people committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    1477ba0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    842aa1f View commit details
    Browse the repository at this point in the history
  3. Lint

    Yard1 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    dd1726f View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Checked CUDA ROCm Compatibility (#15)

    Features
    
    * Auto-code path selection
    * support llama2
    * support squeezellm rocm
    * add documentation amd-installation.rst. Describing how to setup vllm ROCm version
    * format.sh all the code
    * add base amd.Dockerfile
    
    ---------
    
    Co-authored-by: tjtanaa <[email protected]>
    Co-authored-by: kuanfu <[email protected]>
    3 people authored Nov 29, 2023
    Configuration menu
    Copy the full SHA
    9a1781c View commit details
    Browse the repository at this point in the history
  2. merged with latest upstream

    kliuae committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    0f67117 View commit details
    Browse the repository at this point in the history
  3. format code

    kliuae committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    7dbf2d4 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    52ffcf0 View commit details
    Browse the repository at this point in the history
  5. Add rank check

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    6c66b6e View commit details
    Browse the repository at this point in the history
  6. Add example, minor tweaks

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    70eaca6 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    a3f191a View commit details
    Browse the repository at this point in the history
  8. Fix capacity

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    240cee9 View commit details
    Browse the repository at this point in the history
  9. Lint

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    c4d57a5 View commit details
    Browse the repository at this point in the history
  10. Remove rank 128 for now

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    471f25a View commit details
    Browse the repository at this point in the history
  11. Pass to scheduler

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    ccbb4b7 View commit details
    Browse the repository at this point in the history
  12. Add simple scheduler support

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    5a1a0be View commit details
    Browse the repository at this point in the history
  13. Update example

    Yard1 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    1b00e50 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Fix

    Yard1 committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    6bda3c3 View commit details
    Browse the repository at this point in the history
  2. Update tests

    Yard1 committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    de02961 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0afd3c1 View commit details
    Browse the repository at this point in the history
  4. Cleanup

    Yard1 committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    849831e View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Merged changes from vllm main

    kliuae committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    27f0513 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5cce649 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    16d3ccc View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2023

  1. Configuration menu
    Copy the full SHA
    d764f9d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e798632 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0e8129f View commit details
    Browse the repository at this point in the history
  4. Format code

    kliuae committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    2b3821b View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2023

  1. Configuration menu
    Copy the full SHA
    96fa0b2 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. Configuration menu
    Copy the full SHA
    0c8795a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5793f30 View commit details
    Browse the repository at this point in the history
  3. Disable mistral for ROCm

    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    b172cdd View commit details
    Browse the repository at this point in the history
  4. Format code

    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    9cd5b18 View commit details
    Browse the repository at this point in the history
  5. Do not pin memory in WSL

    Yard1 committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    6654033 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. Revert to cuda kernels

    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    b86f88a View commit details
    Browse the repository at this point in the history
  2. Merge remote-tracking branch 'pcmoritz/port-to-rocm'

    Co-authored-by: Philipp Moritz <[email protected]>
    Co-authored-by: Amir Balwel <[email protected]>
    3 people committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    9727ab4 View commit details
    Browse the repository at this point in the history
  3. Port latest kernels to ROCm

    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c4aa2af View commit details
    Browse the repository at this point in the history
  4. Update readme

    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    f8c304e View commit details
    Browse the repository at this point in the history
  5. Cleaned up kernel code

    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    e608c30 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    cf633a7 View commit details
    Browse the repository at this point in the history
  7. Support more vocab sizes

    Yard1 committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    65d1542 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Configuration menu
    Copy the full SHA
    951e225 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    25f9a97 View commit details
    Browse the repository at this point in the history
  3. Updated ROCm warp size

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    e984ada View commit details
    Browse the repository at this point in the history
  4. Format code

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    cc1195f View commit details
    Browse the repository at this point in the history
  5. Check hip from wrapper

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    f92980e View commit details
    Browse the repository at this point in the history
  6. Format code

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    66b4aa1 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    4a0ecb8 View commit details
    Browse the repository at this point in the history
  8. Fixed hip device attribute

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    acf51a8 View commit details
    Browse the repository at this point in the history
  9. Format code

    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4a52977 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    008e92d View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    c328e58 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Restored awq file

    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    23a987a View commit details
    Browse the repository at this point in the history
  2. Format code

    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    8787a4e View commit details
    Browse the repository at this point in the history
  3. Merge latest vllm main

    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5911131 View commit details
    Browse the repository at this point in the history
  4. Updated rocm dockerfile

    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    9fa8075 View commit details
    Browse the repository at this point in the history
  5. Update amd installation guide

    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    81e052d View commit details
    Browse the repository at this point in the history
  6. Update vLLM Documentations (#18)

    Update the vLLM installation procedures on AMD platform.
    Update vLLM documentations.
    tjtanaa authored Dec 7, 2023
    Configuration menu
    Copy the full SHA
    fb8ac26 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Configuration menu
    Copy the full SHA
    98f5487 View commit details
    Browse the repository at this point in the history
  2. Updated setup.py

    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    d90187a View commit details
    Browse the repository at this point in the history
  3. Format code

    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    c840531 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9dba1d8 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. Configuration menu
    Copy the full SHA
    6080fff View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Reuse code

    Yard1 authored Dec 13, 2023
    Configuration menu
    Copy the full SHA
    8566144 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b080e5f View commit details
    Browse the repository at this point in the history
  3. Naming

    Yard1 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    2d72ae5 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Ported to rocm

    kliuae committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    dd0cbca View commit details
    Browse the repository at this point in the history
  2. Added notes in README

    kliuae committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    6e62551 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Added loop unrolling

    kliuae committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    1097aa8 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2023

  1. Merge 'upstream'

    kliuae committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    bc08640 View commit details
    Browse the repository at this point in the history
  2. Remove redundant include

    kliuae committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    4415c92 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2023

  1. Fix merge bugs

    kliuae committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    46678e1 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2023

  1. Configuration menu
    Copy the full SHA
    b6bc40e View commit details
    Browse the repository at this point in the history
  2. Apply feedback from code review

    Yard1 committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    6640a2e View commit details
    Browse the repository at this point in the history
  3. format

    kliuae committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    a9f2ad5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d708898 View commit details
    Browse the repository at this point in the history
  5. Fix

    kliuae committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    1d2ddb7 View commit details
    Browse the repository at this point in the history
  6. Add temporary fixes

    kliuae committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    4512cd1 View commit details
    Browse the repository at this point in the history
  7. Add temporary fixes

    kliuae committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    43bcf2c View commit details
    Browse the repository at this point in the history

Commits on Dec 23, 2023

  1. Fixes

    Yard1 committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    6b2e6a5 View commit details
    Browse the repository at this point in the history
  2. Fixes

    Yard1 committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    4b2224e View commit details
    Browse the repository at this point in the history
  3. Simplify includes

    kliuae committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    dc436ef View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    156e397 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ba7626a View commit details
    Browse the repository at this point in the history
  6. update readme

    kliuae committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    6cc1b42 View commit details
    Browse the repository at this point in the history

Commits on Dec 26, 2023

  1. Configuration menu
    Copy the full SHA
    5539ea9 View commit details
    Browse the repository at this point in the history
  2. Remove unused conversions

    kliuae committed Dec 26, 2023
    Configuration menu
    Copy the full SHA
    52d25ce View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. Add patches for ROCm

    kliuae committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    7ddd344 View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2024

  1. Add rank 128

    kliuae committed Jan 6, 2024
    Configuration menu
    Copy the full SHA
    bf2dea1 View commit details
    Browse the repository at this point in the history