Code

Multi lora cpu#1

Code

Closed

Akshat-Tripathi wants to merge 20 commits intomainkrai/vllm:mainfrom multi_lora_cpukrai/vllm:multi_lora_cpuCopy head branch name to clipboard

+381-90

Commits

Commits on Dec 9, 2024

• Added non-triton SGMV and BGMV ops (not kernels yet)

  Show description for 4a5df02
  Akshat-Tripathi

  committed
  4a5df02

  Copy full SHA for 4a5df02

• Removed extra print

  Show description for 8c45f11
  Akshat-Tripathi

  committed
  8c45f11

  Copy full SHA for 8c45f11

• Minor changes

  Show description for 676991b
  Akshat-Tripathi

  committed
  676991b

  Copy full SHA for 676991b

• Made some minor shape-based fixes to the kernels

  Show description for 3457a73
  Akshat-Tripathi

  committed
  3457a73

  Copy full SHA for 3457a73

• Added multi lora execution to the CPU backend

  Show description for 898e4e4
  Akshat-Tripathi

  committed
  898e4e4

  Copy full SHA for 898e4e4

• Added __init__.py files

  Show description for c537cc9
  Akshat-Tripathi

  committed
  c537cc9

  Copy full SHA for c537cc9

• Changed lora op importing to work irrespective of backend

  Show description for 8a17bc5
  Akshat-Tripathi

  committed
  8a17bc5

  Copy full SHA for 8a17bc5

• Removed comment

  Show description for 733cecc
  Akshat-Tripathi

  committed
  733cecc

  Copy full SHA for 733cecc

• Added multimodal lora support

  Show description for 5b8d04a
  Akshat-Tripathi

  committed
  5b8d04a

  Copy full SHA for 5b8d04a

• Ran formatter

  Show description for 7ad9748
  Akshat-Tripathi

  committed
  7ad9748

  Copy full SHA for 7ad9748

• Changed test backend back to nccl

  Show description for d639b1a
  Akshat-Tripathi

  committed
  d639b1a

  Copy full SHA for d639b1a

• Updated lora tests to work with cpu

  Show description for 56c07d7
  Akshat-Tripathi

  committed
  56c07d7

  Copy full SHA for 56c07d7

• Fixed lora op dtypes

  Show description for 4059ee7
  Akshat-Tripathi

  committed
  4059ee7

  Copy full SHA for 4059ee7

• Updated tests to work with the CPU backend

  Show description for 9473042
  Akshat-Tripathi

  committed
  9473042

  Copy full SHA for 9473042

• Removed redundant .squeeze call in lora ops

  Show description for e00b6bd
  Akshat-Tripathi

  committed
  e00b6bd

  Copy full SHA for e00b6bd

• Stopped skipping some e2e tests

  Show description for 22fc007
  Akshat-Tripathi

  committed
  22fc007

  Copy full SHA for 22fc007

• Fixed case where the lora_b tensor has 4 dims not 3

  Show description for 1272784
  Akshat-Tripathi

  committed
  1272784

  Copy full SHA for 1272784

• Formatted code

  Show description for fb5e02f
  Akshat-Tripathi

  committed
  fb5e02f

  Copy full SHA for fb5e02f

Commits on Dec 10, 2024

• Merge branch 'main' into multi_lora_cpu

  Akshat-Tripathi

  committed
  622c344

  Copy full SHA for 622c344

• Fixed formatting issues

  Show description for c3e9afe
  Akshat-Tripathi

  committed
  c3e9afe

  Copy full SHA for c3e9afe