Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 10 08 #226

Merged
merged 154 commits into from
Oct 9, 2024
Merged

Upstream merge 24 10 08 #226

merged 154 commits into from
Oct 9, 2024

Commits on Sep 26, 2024

  1. [Build/CI] Upgrade to gcc 10 in the base build Docker image (vllm-pro… 

    …ject#8814)
    @tlrmchlsmth
    tlrmchlsmth authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    f70bcca View commit details
    Browse the repository at this point in the history

  2. [Docs] Add README to the build docker image (vllm-project#8825)

    @mgoin
    mgoin authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    520db4d View commit details
    Browse the repository at this point in the history

  3. [CI/Build] Fix missing ci dependencies (vllm-project#8834)

    @fyuan1316
    fyuan1316 authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    68988d4 View commit details
    Browse the repository at this point in the history

  4. [misc][installation] build from source without compilation (vllm-proj… 

    …ect#8818)
    @youkaichao
    youkaichao authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    70de39f View commit details
    Browse the repository at this point in the history

  5. [ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (vllm-pr… 

    …oject#8872)

Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    d9cfbc8 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] Include encoder prompts len to non-stream api usage response (… 

    …vllm-project#8861)
    @Pernekhan
    Pernekhan authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    93d364d View commit details
    Browse the repository at this point in the history

  7. [Misc] Change dummy profiling and BOS fallback warns to log once (vll… 

    …m-project#8820)
    @mgoin
    mgoin authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b28d210 View commit details
    Browse the repository at this point in the history

  8. [Bugfix] Fix print_warning_once's line info (vllm-project#8867)

    @tlrmchlsmth
    tlrmchlsmth authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e2f6f26 View commit details
    Browse the repository at this point in the history

  9. fix validation: Only set tool_choice auto if at least one tool is p… 

    …rovided (vllm-project#8568)
    @chiragjn
    chiragjn authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    ee2da3e View commit details
    Browse the repository at this point in the history

  10. [Bugfix] Fixup advance_step.cu warning (vllm-project#8815)

    @tlrmchlsmth
    tlrmchlsmth authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    71d21c7 View commit details
    Browse the repository at this point in the history

  11. [BugFix] Fix test breakages from transformers 4.45 upgrade (vllm-proj… 

    …ect#8829)
    @njhill
    njhill authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4b377d6 View commit details
    Browse the repository at this point in the history

  12. [Installation] Allow lower versions of FastAPI to maintain Ray 2.9 co… 

    …mpatibility (vllm-project#8764)
    @DarkLight1337
    DarkLight1337 authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    1b49148 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. [Feature] Add support for Llama 3.1 and 3.2 tool use (vllm-project#8343) 

    Signed-off-by: Max de Bayser <[email protected]>
    @maxdebayser
    maxdebayser authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    344cd2b View commit details
    Browse the repository at this point in the history

  2. [Core] renamePromptInputs and inputs (vllm-project#8876)

    @DarkLight1337
    DarkLight1337 authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    3b00b9c View commit details
    Browse the repository at this point in the history

  3. [misc] fix collect env (vllm-project#8894)

    @youkaichao
    youkaichao authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    dc4e3df View commit details
    Browse the repository at this point in the history

  4. [MISC] Fix invalid escape sequence '\' (vllm-project#8830) 

    Signed-off-by: Peter Pan <[email protected]>
    @panpan0000
    panpan0000 authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    0e08875 View commit details
    Browse the repository at this point in the history

  5. [Bugfix][VLM] Fix Fuyu batching inference with max_num_seqs>1 (vllm… 

    …-project#8892)
    @Isotr0py
    Isotr0py authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    6d792d2 View commit details
    Browse the repository at this point in the history

  6. [TPU] Update pallas.py to support trillium (vllm-project#8871)

    @bvrockwell
    bvrockwell authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    8df2dc3 View commit details
    Browse the repository at this point in the history

  7. [torch.compile] use empty tensor instead of None for profiling (vllm-… 

    …project#8875)
    @youkaichao
    youkaichao authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    a9b15c6 View commit details
    Browse the repository at this point in the history

  8. [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear meth… 

    …od (vllm-project#7271)
    @ProExpertProg
    ProExpertProg authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    172d1cd View commit details
    Browse the repository at this point in the history

  9. [Bugfix] fix for deepseek w4a16 (vllm-project#8906) 

    Co-authored-by: mgoin <[email protected]>
    @LucasWilkinson @mgoin
    LucasWilkinson and mgoin authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    c5d5535 View commit details
    Browse the repository at this point in the history

  10. [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path ( 

    vllm-project#8378)

Co-authored-by: Varun Sundar Rabindranath <[email protected]>
    @varun-sundar-rabindranath
    varun-sundar-rabindranath and Varun Sundar Rabindranath authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    c2ec430 View commit details
    Browse the repository at this point in the history

  11. [misc][distributed] add VLLM_SKIP_P2P_CHECK flag (vllm-project#8911)

    @youkaichao
    youkaichao authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    18e60d7 View commit details
    Browse the repository at this point in the history

  12. [Core] Priority-based scheduling in async engine (vllm-project#8850)

    @schoennenbeck
    schoennenbeck authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    bd429f2 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2024

  1. [misc] fix wheel name (vllm-project#8919)

    @youkaichao
    youkaichao authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    d86f6b2 View commit details
    Browse the repository at this point in the history

  2. [Bugfix][Intel] Fix XPU Dockerfile Build (vllm-project#7824) 

    Signed-off-by: tylertitsworth <[email protected]>
Co-authored-by: youkaichao <[email protected]>
    @tylertitsworth @youkaichao
    tylertitsworth and youkaichao authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    260024a View commit details
    Browse the repository at this point in the history

  3. [Misc] Remove vLLM patch of BaichuanTokenizer (vllm-project#8921)

    @DarkLight1337
    DarkLight1337 authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    b0298aa View commit details
    Browse the repository at this point in the history

  4. [Bugfix] Fix code for downloading models from modelscope (vllm-projec… 

    …t#8443)
    @tastelikefeet
    tastelikefeet authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    39d3f8d View commit details
    Browse the repository at this point in the history

  5. [Bugfix] Fix PP for Multi-Step (vllm-project#8887)

    @varun-sundar-rabindranath
    varun-sundar-rabindranath authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    19d02ff View commit details
    Browse the repository at this point in the history

  6. [CI/Build] Update models tests & examples (vllm-project#8874) 

    Co-authored-by: Roger Wang <[email protected]>
    @DarkLight1337 @ywang96
    DarkLight1337 and ywang96 authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    e1a3f5e View commit details
    Browse the repository at this point in the history

  7. [Frontend] Make beam search emulator temperature modifiable (vllm-pro… 

    …ject#8928)

Co-authored-by: Eduard Balzin <[email protected]>
    @nFunctor
    nFunctor and Eduard Balzin authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    090e945 View commit details
    Browse the repository at this point in the history

  8. [Bugfix] Support testing prefill throughput with benchmark_serving.py… 

    … --hf-output-len 1 (vllm-project#8891)
    @heheda12345
    heheda12345 authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    e585b58 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. [doc] organize installation doc and expose per-commit docker (vllm-pr… 

    …oject#8931)
    @youkaichao
    youkaichao authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    cc27644 View commit details
    Browse the repository at this point in the history

  2. [Core] Improve choice of Python multiprocessing method (vllm-project#… 

    …8823)

Signed-off-by: Russell Bryant <[email protected]>
Co-authored-by: youkaichao <[email protected]>
    @russellb @youkaichao
    russellb and youkaichao authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    d153703 View commit details
    Browse the repository at this point in the history

  3. [Bugfix] Block manager v2 with preemption and lookahead slots (vllm-p… 

    …roject#8824)
    @sroy745
    sroy745 authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    5bf8789 View commit details
    Browse the repository at this point in the history

  4. [Bugfix] Fix Marlin MoE act order when is_k_full == False (vllm-proje… 

    …ct#8741)

Co-authored-by: Tyler Michael Smith <[email protected]>
    @ElizaWszola @tlrmchlsmth
    ElizaWszola and tlrmchlsmth authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    d081da0 View commit details
    Browse the repository at this point in the history

  5. [CI/Build] Add test decorator for minimum GPU memory (vllm-project#8925)

    @DarkLight1337
    DarkLight1337 authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    26a68d5 View commit details
    Browse the repository at this point in the history

  6. [Build/CI] Set FETCHCONTENT_BASE_DIR to one location for better cachi… 

    …ng (vllm-project#8930)
    @tlrmchlsmth
    tlrmchlsmth authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    2e7fe7e View commit details
    Browse the repository at this point in the history

  7. [Model] Support Qwen2.5-Math-RM-72B (vllm-project#8896)

    @zhuzilin
    zhuzilin authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    bc2ef1f View commit details
    Browse the repository at this point in the history

  8. [Model][LoRA]LoRA support added for MiniCPMV2.5 (vllm-project#7199)

    @jeejeelee
    jeejeelee authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    3d49776 View commit details
    Browse the repository at this point in the history

  9. [BugFix] Fix seeded random sampling with encoder-decoder models (vllm… 

    …-project#8870)

Co-authored-by: Roger Wang <[email protected]>
    @njhill @ywang96
    njhill and ywang96 authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    31f46a0 View commit details
    Browse the repository at this point in the history

  10. [Misc] Fix typo in BlockSpaceManagerV1 (vllm-project#8944)

    @juncheoll
    juncheoll authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    1fb9c1b View commit details
    Browse the repository at this point in the history

  11. [Frontend] Added support for HF's new continue_final_message parame… 

    …ter (vllm-project#8942)
    @danieljannai21
    danieljannai21 authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    6c9ba48 View commit details
    Browse the repository at this point in the history

  12. [Kernel][Model] Varlen prefill + Prefill chunking support for mamba k… 

    …ernels and Jamba model (vllm-project#8533)
    @mzusman
    mzusman authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    f13a07b View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. [Model] support input embeddings for qwen2vl (vllm-project#8856)

    @whyiug
    whyiug authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    e01ab59 View commit details
    Browse the repository at this point in the history

  2. [Misc][CI/Build] Include cv2 via mistral_common[opencv] (vllm-pro… 

    …ject#8951)
    @ywang96
    ywang96 authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    b6d7392 View commit details
    Browse the repository at this point in the history

  3. [Model][LoRA]LoRA support added for MiniCPMV2.6 (vllm-project#8943) 

    Co-authored-by: DarkLight1337 <[email protected]>
    @jeejeelee @DarkLight1337
    jeejeelee and DarkLight1337 authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    8e60afa View commit details
    Browse the repository at this point in the history

  4. [Model] Expose InternVL2 max_dynamic_patch as a mm_processor_kwarg (v… 

    …llm-project#8946)
    @Isotr0py
    Isotr0py authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    2ae25f7 View commit details
    Browse the repository at this point in the history

  5. [Core] Make scheduling policy settable via EngineArgs (vllm-project#8956 

    )
    @schoennenbeck
    schoennenbeck authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    be76e5a View commit details
    Browse the repository at this point in the history

  6. [Misc] Adjust max_position_embeddings for LoRA compatibility (vllm-pr… 

    …oject#8957)
    @jeejeelee
    jeejeelee authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    1cabfce View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. [ci] Add CODEOWNERS for test directories (vllm-project#8795) 

    Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    1425a1b View commit details
    Browse the repository at this point in the history

  2. [CI][SpecDecode] Fix spec decode tests, use flash attention backend f… 

    …or spec decode CI tests. (vllm-project#8975)
    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    bce3244 View commit details
    Browse the repository at this point in the history

  3. [Frontend][Core] Move guided decoding params into sampling params (vl… 

    …lm-project#8252)

Signed-off-by: Joe Runde <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
    @joerunde @njhill
    joerunde and njhill authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    062c89e View commit details
    Browse the repository at this point in the history

  4. [CI/Build] Fix machete generated kernel files ordering (vllm-project#… 

    …8976)

Signed-off-by: kevin <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
    @khluu @comaniac
    khluu and comaniac authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    aaccca2 View commit details
    Browse the repository at this point in the history

  5. [torch.compile] fix tensor alias (vllm-project#8982)

    @youkaichao
    youkaichao authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    7da2487 View commit details
    Browse the repository at this point in the history

  6. [Misc] add process_weights_after_loading for DummyLoader (vllm-projec… 

    …t#8969)
    @divakar-amd
    divakar-amd authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    82f3937 View commit details
    Browse the repository at this point in the history

  7. [Bugfix] Fix Fuyu tensor parallel inference (vllm-project#8986)

    @Isotr0py
    Isotr0py authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    bc4eb65 View commit details
    Browse the repository at this point in the history

  8. [Bugfix] Fix Token IDs Reference for MiniCPM-V When Images are Provid… 

    …ed With No Placeholders (vllm-project#8991)

Signed-off-by: Alex-Brooks <[email protected]>
    @alex-jw-brooks
    alex-jw-brooks authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    1fe0a42 View commit details
    Browse the repository at this point in the history

  9. [Core] [Frontend] Priority scheduling for embeddings and in the OpenA… 

    …I-API (vllm-project#8965)
    @schoennenbeck
    schoennenbeck authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    35bd215 View commit details
    Browse the repository at this point in the history

  10. [Doc] Update list of supported models (vllm-project#8987)

    @DarkLight1337
    DarkLight1337 authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    4f341bd View commit details
    Browse the repository at this point in the history

  11. Update benchmark_serving.py to read and write json-datasets, results … 

    …in UTF8, for better compatibility with Windows (vllm-project#8997)
    @vlsav
    vlsav authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    22f5851 View commit details
    Browse the repository at this point in the history

  12. [Spec Decode] (1/2) Remove batch expansion (vllm-project#8839)

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    1570203 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. [Core] Combined support for multi-step scheduling, chunked prefill & … 

    …prefix caching (vllm-project#8804)

Co-authored-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Andrew Feldman <[email protected]>
    @afeldman-nm @abf149
    3 people authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    563649a View commit details
    Browse the repository at this point in the history

  2. [Misc] Update Default Image Mapper Error Log (vllm-project#8977) 

    Signed-off-by: Alex-Brooks <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
    @alex-jw-brooks @ywang96
    alex-jw-brooks and ywang96 authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    7f60520 View commit details
    Browse the repository at this point in the history

  3. [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (vllm-project#8645) 

    Co-authored-by: Varun Sundar Rabindranath <[email protected]>
    @varun-sundar-rabindranath
    varun-sundar-rabindranath and Varun Sundar Rabindranath authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    afb050b View commit details
    Browse the repository at this point in the history

  4. [OpenVINO] Enable GPU support for OpenVINO vLLM backend (vllm-project… 

    …#8192)
    @sshlyapn
    sshlyapn authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    f58d4fc View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. [Model] Adding Granite MoE. (vllm-project#8206) 

    Co-authored-by: Nick Hill <[email protected]>
    @shawntan @njhill
    shawntan and njhill authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    19f0d25 View commit details
    Browse the repository at this point in the history

  2. [Doc] Update Granite model docs (vllm-project#9025)

    @njhill
    njhill authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    18c2e30 View commit details
    Browse the repository at this point in the history

  3. [Bugfix] example template should not add parallel_tool_prompt if tool… 

    …s is none (vllm-project#9007)
    @tjohnson31415
    tjohnson31415 authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    19a4dd0 View commit details
    Browse the repository at this point in the history

  4. [Misc] log when using default MoE config (vllm-project#8971)

    @divakar-amd
    divakar-amd authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    01843c8 View commit details
    Browse the repository at this point in the history

  5. [BugFix] Enforce Mistral ToolCall id constraint when using the Mistra… 

    …l tool call parser (vllm-project#9020)
    @gcalmettes
    gcalmettes authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    83caf35 View commit details
    Browse the repository at this point in the history

  6. [Core] Make BlockSpaceManagerV2 the default BlockManager to use. (vll… 

    …m-project#8678)
    @sroy745
    sroy745 authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    f5d72b2 View commit details
    Browse the repository at this point in the history

  7. [Frontend] [Neuron] Parse literals out of override-neuron-config (vll… 

    …m-project#8959)

Co-authored-by: Jerzy Zagorski <[email protected]>
    @xendo
    xendo and Jerzy Zagorski authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    63e3993 View commit details
    Browse the repository at this point in the history

  8. [misc] add forward context for attention (vllm-project#9029)

    @youkaichao
    youkaichao authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    9aaf14c View commit details
    Browse the repository at this point in the history

  9. Fix failing spec decode test (vllm-project#9054)

    @sroy745
    sroy745 authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    91add85 View commit details
    Browse the repository at this point in the history

  10. [Bugfix] Weight loading fix for OPT model (vllm-project#9042) 

    Co-authored-by: dvres <[email protected]>
    @domenVres
    domenVres and dvres authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    2838d6b View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. [Frontend][Feature] support tool calling for internlm/internlm2_5-7b-… 

    …chat model (vllm-project#8405)
    @sydnash
    sydnash authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    3dbb215 View commit details
    Browse the repository at this point in the history

  2. [CI/Build] Per file CUDA Archs (improve wheel size and dev build time… 

    …s) (vllm-project#8845)
    @LucasWilkinson
    LucasWilkinson authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    aeb37c2 View commit details
    Browse the repository at this point in the history

  3. [Misc] Enable multi-step output streaming by default (vllm-project#9047)

    @mgoin
    mgoin authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    303d447 View commit details
    Browse the repository at this point in the history

  4. [Models] Add remaining model PP support (vllm-project#7168) 

    Signed-off-by: Muralidhar Andoorveedu <[email protected]>
Signed-off-by: Murali Andoorveedu <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
    @andoorve @DarkLight1337
    andoorve and DarkLight1337 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    0f6d7a9 View commit details
    Browse the repository at this point in the history

  5. [Misc] Move registry to its own file (vllm-project#9064)

    @DarkLight1337
    DarkLight1337 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    0e36fd4 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] Reshape the dimensions of the input image embeddings in Qwen… 

    …2VL (vllm-project#9071)
    @whyiug
    whyiug authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    3d826d2 View commit details
    Browse the repository at this point in the history

  7. [Bugfix] Flash attention arches not getting set properly (vllm-projec… 

    …t#9062)
    @LucasWilkinson
    LucasWilkinson authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    22482e4 View commit details
    Browse the repository at this point in the history

  8. [Model] add a bunch of supported lora modules for mixtral (vllm-proje… 

    …ct#9008)

Signed-off-by: Prashant Gupta <[email protected]>
    @prashantgupta24
    prashantgupta24 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    9ade8bb View commit details
    Browse the repository at this point in the history

  9. Remove AMD Ray Summit Banner (vllm-project#9075)

    @simon-mo
    simon-mo authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    36eecfb View commit details
    Browse the repository at this point in the history

  10. [Hardware][PowerPC] Make oneDNN dependency optional for Power (vllm-p… 

    …roject#9039)

Signed-off-by: Varad Ahirwadkar <[email protected]>
    @varad-ahirwadkar
    varad-ahirwadkar authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    e5dc713 View commit details
    Browse the repository at this point in the history

  11. [Core][VLM] Test registration for OOT multimodal models (vllm-project… 

    …#8717)

Co-authored-by: DarkLight1337 <[email protected]>
    @ywang96 @DarkLight1337
    ywang96 and DarkLight1337 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    26aa325 View commit details
    Browse the repository at this point in the history

  12. Adds truncate_prompt_tokens param for embeddings creation (vllm-proje… 

    …ct#8999)

Signed-off-by: Flavia Beo <[email protected]>
    @flaviabeo
    flaviabeo authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    0dcc8cb View commit details
    Browse the repository at this point in the history

  13. [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (… 

    …vllm-project#8973)

Co-authored-by: Dipika <[email protected]>
Co-authored-by: Dipika Sikka <[email protected]>
    @ElizaWszola @dsikka
    3 people authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    05d6864 View commit details
    Browse the repository at this point in the history

  14. [CI] Update performance benchmark: upgrade trt-llm to r24.07, and add… 

    … SGLang (vllm-project#7412)
    @KuntaiDu
    KuntaiDu authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    fbb7442 View commit details
    Browse the repository at this point in the history

  15. [Misc] Improved prefix cache example (vllm-project#9077)

    @Imss27
    Imss27 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    05c531b View commit details
    Browse the repository at this point in the history

  16. [Misc] Add random seed for prefix cache benchmark (vllm-project#9081)

    @Imss27
    Imss27 authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    0cc566c View commit details
    Browse the repository at this point in the history

  17. [Misc] Fix CI lint (vllm-project#9085)

    @comaniac
    comaniac authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    27302dd View commit details
    Browse the repository at this point in the history

  18. [Hardware][Neuron] Add on-device sampling support for Neuron (vllm-pr… 

    …oject#8746)

Co-authored-by: Ashraf Mahgoub <[email protected]>
    @chongmni-aws @aws-aymahg
    chongmni-aws and aws-aymahg authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    cc90419 View commit details
    Browse the repository at this point in the history

  19. [torch.compile] improve allreduce registration (vllm-project#9061)

    @youkaichao
    youkaichao authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    663874e View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2024

  1. [Doc] Update README.md with Ray summit slides (vllm-project#9088)

    @zhuohan123
    zhuohan123 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    a95354a View commit details
    Browse the repository at this point in the history

  2. [Bugfix] use blockmanagerv1 for encoder-decoder (vllm-project#9084) 

    Co-authored-by: Roger Wang <[email protected]>
    @heheda12345 @ywang96
    heheda12345 and ywang96 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    dac914b View commit details
    Browse the repository at this point in the history

  3. [Bugfix] Fixes Phi3v & Ultravox Multimodal EmbeddingInputs (vllm-proj… 

    …ect#8979)
    @hhzhang16
    hhzhang16 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    53b3a33 View commit details
    Browse the repository at this point in the history

  4. [Model] Support Gemma2 embedding model (vllm-project#9004)

    @xyang16
    xyang16 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    15986f5 View commit details
    Browse the repository at this point in the history

  5. [Bugfix] Deprecate registration of custom configs to huggingface (vll… 

    …m-project#9083)
    @heheda12345
    heheda12345 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    cfadb9c View commit details
    Browse the repository at this point in the history

  6. [Bugfix] Fix order of arguments matters in config.yaml (vllm-project#… 

    …8960)
    @Imss27
    Imss27 authored Oct 5, 2024
    Configuration menu
    Copy the full SHA
    5df1834 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. [core] use forward context for flash infer (vllm-project#9097)

    @youkaichao
    youkaichao authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    f4dd830 View commit details
    Browse the repository at this point in the history

  2. [Bugfix] Fix try-catch conditions to import correct Flash Attention B… 

    …ackend in Draft Model (vllm-project#9101)
    @tjtanaa
    tjtanaa authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    23fea87 View commit details
    Browse the repository at this point in the history

  3. [Frontend] API support for beam search (vllm-project#9087) 

    Co-authored-by: youkaichao <[email protected]>
    @LunrEclipse @youkaichao
    LunrEclipse and youkaichao authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    168cab6 View commit details
    Browse the repository at this point in the history

  4. [Misc] Remove user-facing error for removed VLM args (vllm-project#9104)

    @DarkLight1337
    DarkLight1337 authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    f22619f View commit details
    Browse the repository at this point in the history

  5. [Model] PP support for embedding models and update docs (vllm-project… 

    …#9090)

Co-authored-by: Roger Wang <[email protected]>
    @DarkLight1337 @ywang96
    DarkLight1337 and ywang96 authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    b22b798 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] fix tool_parser error handling when serve a model not suppor… 

    …t it (vllm-project#8709)
    @liuyanyi
    liuyanyi authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    fdf59d3 View commit details
    Browse the repository at this point in the history

  7. [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step s… 

    …cheduling (vllm-project#9038)

Co-authored-by: Varun Sundar Rabindranath <[email protected]>
    @varun-sundar-rabindranath
    varun-sundar-rabindranath and Varun Sundar Rabindranath authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    cb3b2b9 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. [Bugfix][Hardware][CPU] Fix CPU model input for decode (vllm-project#… 

    …9044)
    @Isotr0py
    Isotr0py authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    487678d View commit details
    Browse the repository at this point in the history

  2. [BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (vllm-pr… 

    …oject#9103)
    @sroy745
    sroy745 authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    c8f26bb View commit details
    Browse the repository at this point in the history

  3. [core] remove beam search from the core (vllm-project#9105)

    @youkaichao
    youkaichao authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    18b296f View commit details
    Browse the repository at this point in the history

  4. [Model] Explicit interface for vLLM models and support OOT embedding … 

    …models (vllm-project#9108)
    @DarkLight1337
    DarkLight1337 authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    8c6de96 View commit details
    Browse the repository at this point in the history

  5. [Hardware][CPU] Cross-attention and Encoder-Decoder models support on… 

    … CPU backend (vllm-project#9089)
    @Isotr0py
    Isotr0py authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    4f95ffe View commit details
    Browse the repository at this point in the history

  6. [Core] Refactor GGUF parameters packing and forwarding (vllm-project#… 

    …8859)
    @Isotr0py
    Isotr0py authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    f19da64 View commit details
    Browse the repository at this point in the history

  7. [Model] Support NVLM-D and fix QK Norm in InternViT (vllm-project#9045) 

    Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
    @DarkLight1337 @ywang96 @Isotr0py
    3 people authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    151ef4e View commit details
    Browse the repository at this point in the history

  8. [Doc]: Add deploying_with_k8s guide (vllm-project#8451)

    @haitwang-cloud
    haitwang-cloud authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    93cf74a View commit details
    Browse the repository at this point in the history

  9. [CI/Build] Add linting for github actions workflows (vllm-project#7876) 

    Signed-off-by: Russell Bryant <[email protected]>
    @russellb
    russellb authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e0dbdb0 View commit details
    Browse the repository at this point in the history

  10. [Doc] Include performance benchmark in README (vllm-project#9135)

    @KuntaiDu
    KuntaiDu authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    c0d9a98 View commit details
    Browse the repository at this point in the history

  11. [misc] fix comment and variable name (vllm-project#9139)

    @youkaichao
    youkaichao authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    fa45513 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Add Slack to README (vllm-project#9137)

    @simon-mo
    simon-mo authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8eeb857 View commit details
    Browse the repository at this point in the history

  2. [misc] update utils to support comparing multiple settings (vllm-proj… 

    …ect#9140)
    @youkaichao
    youkaichao authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    04c12f8 View commit details
    Browse the repository at this point in the history

  3. [Intel GPU] Fix xpu decode input (vllm-project#9145)

    @jikunshang
    jikunshang authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    80b57f0 View commit details
    Browse the repository at this point in the history

  4. [misc] improve ux on readme (vllm-project#9147)

    @youkaichao
    youkaichao authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e1faa2a View commit details
    Browse the repository at this point in the history

  5. [Frontend] API support for beam search for MQLLMEngine (vllm-project#… 

    …9117)
    @LunrEclipse
    LunrEclipse authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8c74622 View commit details
    Browse the repository at this point in the history

  6. [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (v… 

    …llm-project#9131)

Signed-off-by: Alex-Brooks <[email protected]>
    @alex-jw-brooks
    alex-jw-brooks authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a3691b6 View commit details
    Browse the repository at this point in the history

  7. Factor out common weight loading code

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1992aa8 View commit details
    Browse the repository at this point in the history

  8. Fix EAGLE model loading

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e81645d View commit details
    Browse the repository at this point in the history

  9. [Frontend] Add Early Validation For Chat Template / Tool Call Parser (v… 

    …llm-project#9151)

Signed-off-by: Alex-Brooks <[email protected]>
    @alex-jw-brooks
    alex-jw-brooks authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    069d3bd View commit details
    Browse the repository at this point in the history

  10. Improve efficiency

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4ef043b View commit details
    Browse the repository at this point in the history

  11. Rename

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e723680 View commit details
    Browse the repository at this point in the history

  12. Update LLaVA-NeXT-Video

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c60e921 View commit details
    Browse the repository at this point in the history

  13. [CI/Build] Add examples folder into Docker image so that we can lever… 

    …age the templates*.jinja when serving models (vllm-project#8758)

Signed-off-by: Peter Pan <[email protected]>
    @panpan0000
    panpan0000 authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    cfba685 View commit details
    Browse the repository at this point in the history

  14. [Bugfix] fix OpenAI API server startup with --disable-frontend-multip… 

    …rocessing (vllm-project#8537)
    @dtrifiro
    dtrifiro authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9a94ca4 View commit details
    Browse the repository at this point in the history

  15. Automatic loading and save memory

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9f12890 View commit details
    Browse the repository at this point in the history

  16. Rename

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    10b5b0e View commit details
    Browse the repository at this point in the history

  17. Update docstring

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ce08df5 View commit details
    Browse the repository at this point in the history

  18. Simplify

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    df687ac View commit details
    Browse the repository at this point in the history

  19. Cleanup

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    98bf417 View commit details
    Browse the repository at this point in the history

  20. Fully enable recursive loading

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    decc7a4 View commit details
    Browse the repository at this point in the history

  21. Clarify

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e59201a View commit details
    Browse the repository at this point in the history

  22. [Doc] Update vlm.rst to include an example on videos (vllm-project#9155) 

    Co-authored-by: Cyrus Leung <[email protected]>
    @sayakpaul @DarkLight1337
    sayakpaul and DarkLight1337 authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1874c6a View commit details
    Browse the repository at this point in the history

  23. Fix incorrect semantics

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f538ab9 View commit details
    Browse the repository at this point in the history

  24. Move function

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f077865 View commit details
    Browse the repository at this point in the history

  25. Update error message

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    56e4a33 View commit details
    Browse the repository at this point in the history

  26. Fix Ultravox loading

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    85c63c8 View commit details
    Browse the repository at this point in the history

  27. spacing

    @DarkLight1337
    DarkLight1337 committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    42a3253 View commit details
    Browse the repository at this point in the history

  28. Merge remote-tracking branch 'upstream/main'

    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b21ccdf View commit details
    Browse the repository at this point in the history

  29. Merge remote-tracking branch 'upstream/main' into main

    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e5a7def View commit details
    Browse the repository at this point in the history

  30. Merge remote-tracking branch 'upstream/fix-weight-loading' into main

    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3e72cae View commit details
    Browse the repository at this point in the history

  31. Merge remote-tracking branch 'origin/main' into upstream_merge_24_10_08

    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    674b2a5 View commit details
    Browse the repository at this point in the history

  32. [Doc] Improve contributing and installation documentation (vllm-proje… 

    …ct#9132)

Signed-off-by: Rafael Vasquez <[email protected]>
    @rafvasq
    rafvasq authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    de24046 View commit details
    Browse the repository at this point in the history

  33. Fix server

    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    390efcb View commit details
    Browse the repository at this point in the history

  34. [Bugfix] Try to handle older versions of pytorch (vllm-project#9086)

    @bnellnm
    bnellnm authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bd37b9f View commit details
    Browse the repository at this point in the history

  35. Merge remote-tracking branch 'upstream/main' into upstream_merge_24_1… 

    …0_08
    @gshtras
    gshtras committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8fa419f View commit details
    Browse the repository at this point in the history