Upstream merge 24 10 21 #240

gshtras · 2024-10-21T21:17:51Z

No description provided.

…oject#8957)

Signed-off-by: kevin <[email protected]>

…or spec decode CI tests. (vllm-project#8975)

…lm-project#8252) Signed-off-by: Joe Runde <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…8976) Signed-off-by: kevin <[email protected]> Co-authored-by: Cody Yu <[email protected]>

…t#8969)

…ed With No Placeholders (vllm-project#8991) Signed-off-by: Alex-Brooks <[email protected]>

…I-API (vllm-project#8965)

…in UTF8, for better compatibility with Windows (vllm-project#8997)

…prefix caching (vllm-project#8804) Co-authored-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Andrew Feldman <[email protected]>

Signed-off-by: Alex-Brooks <[email protected]> Co-authored-by: Roger Wang <[email protected]>

Co-authored-by: Varun Sundar Rabindranath <[email protected]>

…#8192)

Co-authored-by: Nick Hill <[email protected]>

…s is none (vllm-project#9007)

…l tool call parser (vllm-project#9020)

…m-project#8678)

…m-project#8959) Co-authored-by: Jerzy Zagorski <[email protected]>

Co-authored-by: dvres <[email protected]>

…chat model (vllm-project#8405)

…s) (vllm-project#8845)

…r cores (vllm-project#9497) Signed-off-by: Thomas Parnell <[email protected]> Co-authored-by: Chih-Chieh Yang <[email protected]> Co-authored-by: Cody Yu <[email protected]>

)

Signed-off-by: Joe Runde <[email protected]>

…ly (vllm-project#9521)

Signed-off-by: Joe Runde <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

…lm-project#9514)

…ger (vllm-project#9530)

…llm-project#9520)

…ect#9403)

Co-authored-by: Nick Hill <[email protected]>

Signed-off-by: Thomas Parnell <[email protected]>

…m-project#9492) Signed-off-by: Varad Ahirwadkar <[email protected]>

…ect#9518)

…0_21

maleksan85

If perf tests are well, please land.

jeejeelee and others added 30 commits September 30, 2024 12:57

[Misc] Adjust max_position_embeddings for LoRA compatibility (vllm-pr…

1cabfce

…oject#8957)

[ci] Add CODEOWNERS for test directories (vllm-project#8795)

1425a1b

Signed-off-by: kevin <[email protected]>

[CI][SpecDecode] Fix spec decode tests, use flash attention backend f…

bce3244

…or spec decode CI tests. (vllm-project#8975)

[Frontend][Core] Move guided decoding params into sampling params (vl…

062c89e

…lm-project#8252) Signed-off-by: Joe Runde <[email protected]> Co-authored-by: Nick Hill <[email protected]>

[CI/Build] Fix machete generated kernel files ordering (vllm-project#…

aaccca2

…8976) Signed-off-by: kevin <[email protected]> Co-authored-by: Cody Yu <[email protected]>

[torch.compile] fix tensor alias (vllm-project#8982)

7da2487

[Misc] add process_weights_after_loading for DummyLoader (vllm-projec…

82f3937

…t#8969)

[Bugfix] Fix Fuyu tensor parallel inference (vllm-project#8986)

bc4eb65

[Bugfix] Fix Token IDs Reference for MiniCPM-V When Images are Provid…

1fe0a42

…ed With No Placeholders (vllm-project#8991) Signed-off-by: Alex-Brooks <[email protected]>

[Core] [Frontend] Priority scheduling for embeddings and in the OpenA…

35bd215

…I-API (vllm-project#8965)

[Doc] Update list of supported models (vllm-project#8987)

4f341bd

Update benchmark_serving.py to read and write json-datasets, results …

22f5851

…in UTF8, for better compatibility with Windows (vllm-project#8997)

[Spec Decode] (1/2) Remove batch expansion (vllm-project#8839)

1570203

[Core] Combined support for multi-step scheduling, chunked prefill & …

563649a

…prefix caching (vllm-project#8804) Co-authored-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Andrew Feldman <[email protected]>

[Misc] Update Default Image Mapper Error Log (vllm-project#8977)

7f60520

Signed-off-by: Alex-Brooks <[email protected]> Co-authored-by: Roger Wang <[email protected]>

[Core] CUDA Graphs for Multi-Step + Chunked-Prefill (vllm-project#8645)

afb050b

Co-authored-by: Varun Sundar Rabindranath <[email protected]>

[OpenVINO] Enable GPU support for OpenVINO vLLM backend (vllm-project…

f58d4fc

…#8192)

[Model] Adding Granite MoE. (vllm-project#8206)

19f0d25

Co-authored-by: Nick Hill <[email protected]>

[Doc] Update Granite model docs (vllm-project#9025)

18c2e30

[Bugfix] example template should not add parallel_tool_prompt if tool…

19a4dd0

…s is none (vllm-project#9007)

[Misc] log when using default MoE config (vllm-project#8971)

01843c8

[BugFix] Enforce Mistral ToolCall id constraint when using the Mistra…

83caf35

…l tool call parser (vllm-project#9020)

[Core] Make BlockSpaceManagerV2 the default BlockManager to use. (vll…

f5d72b2

…m-project#8678)

[Frontend] [Neuron] Parse literals out of override-neuron-config (vll…

63e3993

…m-project#8959) Co-authored-by: Jerzy Zagorski <[email protected]>

[misc] add forward context for attention (vllm-project#9029)

9aaf14c

Fix failing spec decode test (vllm-project#9054)

91add85

[Bugfix] Weight loading fix for OPT model (vllm-project#9042)

2838d6b

Co-authored-by: dvres <[email protected]>

[Frontend][Feature] support tool calling for internlm/internlm2_5-7b-…

3dbb215

…chat model (vllm-project#8405)

[CI/Build] Per file CUDA Archs (improve wheel size and dev build time…

aeb37c2

…s) (vllm-project#8845)

[Misc] Enable multi-step output streaming by default (vllm-project#9047)

303d447

tdoublep and others added 23 commits October 18, 2024 17:55

[Kernel] Add env variable to force flashinfer backend to enable tenso…

0c9a525

…r cores (vllm-project#9497) Signed-off-by: Thomas Parnell <[email protected]> Co-authored-by: Chih-Chieh Yang <[email protected]> Co-authored-by: Cody Yu <[email protected]>

[Bugfix] Fix offline mode when using mistral_common (vllm-project#9457

337ed76

)

🐛 fix torch memory profiling (vllm-project#9516)

380e186

Signed-off-by: Joe Runde <[email protected]>

[Frontend] Avoid creating guided decoding LogitsProcessor unnecessari…

1325872

…ly (vllm-project#9521)

[Doc] update gpu-memory-utilization flag docs (vllm-project#9507)

82c2515

Signed-off-by: Joe Runde <[email protected]>

[CI/Build] Add error matching for ruff output (vllm-project#9513)

dfd951e

Signed-off-by: Russell Bryant <[email protected]>

[CI/Build] Configure matcher for actionlint workflow (vllm-project#9511)

85dc92f

Signed-off-by: Russell Bryant <[email protected]>

[Frontend] Support simpler image input format (vllm-project#9478)

c5eea3c

[Bugfix] Fix missing task for speculative decoding (vllm-project#9524)

263d8ee

[Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (vl…

8e3e7f2

…lm-project#9514)

[Bugfix] Pass json-schema to GuidedDecodingParams and make test stron…

5b59fe0

…ger (vllm-project#9530)

[Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (v…

962d2c6

…llm-project#9520)

[Kernel] Support sliding window in flash attention backend (vllm-proj…

4fa3e33

…ect#9403)

[Frontend][Misc] Goodput metric support (vllm-project#9338)

855e0e6

[CI/Build] Split up decoder-only LM tests (vllm-project#9488)

696b01a

Co-authored-by: Nick Hill <[email protected]>

[Doc] Consistent naming of attention backends (vllm-project#9498)

496e991

Signed-off-by: Thomas Parnell <[email protected]>

[Model] FalconMamba Support (vllm-project#9325)

f6b9729

[Bugfix][Misc]: fix graph capture for decoder (vllm-project#9549)

8ca8954

[BugFix] Use correct python3 binary in Docker.ppc64le entrypoint (vll…

ec6bd6c

…m-project#9492) Signed-off-by: Varad Ahirwadkar <[email protected]>

[Model][Bugfix] Fix batching with multi-image in PixtralHF (vllm-proj…

5241aa1

…ect#9518)

Merge remote-tracking branch 'origin/main' into upstream_merge_24_10_21

6e79dcf

Merge remote-tracking branch 'upstream/main' into upstream_merge_24_1…

b10dad1

…0_21

yapf

634d9b0

maleksan85 approved these changes Oct 21, 2024

View reviewed changes

gshtras added 4 commits October 22, 2024 10:37

Merge branch 'main' into upstream_merge_24_10_21

af76c9d

Merge branch 'main' into upstream_merge_24_10_21

a594c0c

Merge branch 'main' into upstream_merge_24_10_21

87e3970

Merge branch 'main' into upstream_merge_24_10_21

be448fb

gshtras merged commit 2a3f461 into main Oct 23, 2024
17 of 18 checks passed

gshtras deleted the upstream_merge_24_10_21 branch October 23, 2024 15:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream merge 24 10 21 #240

Upstream merge 24 10 21 #240

gshtras commented Oct 21, 2024

maleksan85 left a comment

Upstream merge 24 10 21 #240

Upstream merge 24 10 21 #240

Conversation

gshtras commented Oct 21, 2024

maleksan85 left a comment

Choose a reason for hiding this comment