Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support sending additional outputs from vLLM inference #70

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

feat: Support sending additional outputs from vLLM inference #70

wants to merge 12 commits into from

Commits on Nov 1, 2024

  1. Add additional outputs and their input switches to auto complete 

    * [WIP] Add additional outputs to auto complete

* [WIP] Use individual input tensor to control per additional output

* [WIP] Parse additional output flags from request
    @kthui
    kthui committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    10a5b94 View commit details
    Browse the repository at this point in the history

  2. chore: Refactor generate function

    @kthui
    kthui committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    892f0d0 View commit details
    Browse the repository at this point in the history

  3. Add additional outputs to response

    @kthui
    kthui committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    58ee481 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2024

  1. Add test for additional outputs 

    * Add additional outputs test

* Update copyright

* Some test enhancement and notes
    @kthui
    kthui committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    5e605ca View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Add docs for additonal outputs

    @kthui
    kthui committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    f35e9c4 View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. chore: Unify vLLM test names

    @kthui
    kthui committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    e6e6404 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Switch to pytest

    @kthui
    kthui committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    44edd6e View commit details
    Browse the repository at this point in the history

  2. pytest to dump additional outputs

    @kthui
    kthui committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    1773dea View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. Rename output_* to return_*

    @kthui
    kthui committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    29099df View commit details
    Browse the repository at this point in the history

  2. Return token ids instead of number of token ids

    @kthui
    kthui committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    457eeaa View commit details
    Browse the repository at this point in the history

  3. Revert "Return token ids instead of number of token ids" 

    This reverts commit 457eeaa.
    @kthui
    kthui committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    5e9b09f View commit details
    Browse the repository at this point in the history

  4. Rename num_token_ids to num_output_tokens

    @kthui
    kthui committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    dae3c13 View commit details
    Browse the repository at this point in the history