Code

misc: attention kernel refactoring#339

Code

Merged

guocuimi merged 13 commits intomainvectorch-ai/ScaleLLM:mainfrom refactor_flashinfervectorch-ai/ScaleLLM:refactor_flashinferCopy head branch name to clipboardOct 23, 2024

+971-615

Commits

Commits on Oct 18, 2024

• feat: added flash infer attention handler

  guocuimi

  committed
  2abc2da

  Copy full SHA for 2abc2da

• read code and add comments

  guocuimi

  committed
  04daa6e

  Copy full SHA for 04daa6e

• rename layout x,y,z to m,k,n

  guocuimi

  committed
  0d0f636

  Copy full SHA for 0d0f636

• fix ut failure

  guocuimi

  committed
  6adca6e

  Copy full SHA for 6adca6e

• more comments

  guocuimi

  committed
  bc79f58

  Copy full SHA for bc79f58

• reorder code

  guocuimi

  committed
  c46c594

  Copy full SHA for c46c594

• avoid using advance_offset_by for s_mem offset

  guocuimi

  committed
  351ceb2

  Copy full SHA for 351ceb2

• use all warps to load q

  guocuimi

  committed
  9533bce

  Copy full SHA for 9533bce

• use all warps to write o

  guocuimi

  committed
  da3662a

  Copy full SHA for da3662a

• add sync before shared memory access

  guocuimi

  committed
  8a8f246

  Copy full SHA for 8a8f246

• clean warnings

  guocuimi

  committed
  6cc1bce

  Copy full SHA for 6cc1bce

Commits on Oct 19, 2024

• replace -inf with -5e4

  guocuimi

  committed
  10f024b

  Copy full SHA for 10f024b

• fix tests

  guocuimi

  committed
  a347c81

  Copy full SHA for a347c81