Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: Refactor nitro with cortext.tensorrtllm engine #37

Closed
wants to merge 78 commits into from
Closed

refactor: Refactor nitro with cortext.tensorrtllm engine #37

wants to merge 78 commits into from

Commits on Mar 8, 2024

  1. Initial rewrite of README

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    7c4d20e View commit details
    Browse the repository at this point in the history

  2. Add note on model field

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    50f9130 View commit details
    Browse the repository at this point in the history

  3. docs: add to readme

    @0xSage
    0xSage committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    7cd0e68 View commit details
    Browse the repository at this point in the history

  4. docs: refactor build from source instructions

    @0xSage
    0xSage committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    078a20a View commit details
    Browse the repository at this point in the history

  5. Add Package details

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    9166a7a View commit details
    Browse the repository at this point in the history

  6. init commit: add nitro

    @tikikun
    tikikun committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    110445e View commit details
    Browse the repository at this point in the history

  7. add build nitro

    @tikikun
    tikikun committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    faa7bc6 View commit details
    Browse the repository at this point in the history

  8. build nitro and not tamper nvidia codebase

    @tikikun
    tikikun committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    8031776 View commit details
    Browse the repository at this point in the history

  9. Add to README

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    a842224 View commit details
    Browse the repository at this point in the history

  10. Merge pull request #3 from janhq/update-readme 

    WIP: Draft README for `nitro-tensorrt-llm`
    @dan-homebrew
    dan-homebrew authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    547395e View commit details
    Browse the repository at this point in the history

  11. chore: resolve merge conflict

    @0xSage
    0xSage committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6a8b452 View commit details
    Browse the repository at this point in the history

  12. docs: add prereqs

    @0xSage
    0xSage committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    92e8451 View commit details
    Browse the repository at this point in the history

  13. Remove unnecessary TODOs

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    061d2c5 View commit details
    Browse the repository at this point in the history

  14. Merge pull request #5 from janhq/nitro-tensorrt-llm 

    [Release] Implement Nitro on top of TensorRT-LLM C++ repo
    @tikikun
    tikikun authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    a8faedf View commit details
    Browse the repository at this point in the history

  15. Merge Nicole's changes

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    85f678d View commit details
    Browse the repository at this point in the history

  16. docs: add instructions for pointing jan fe to the engine

    @0xSage
    0xSage committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    20858f2 View commit details
    Browse the repository at this point in the history

  17. Merge pull request #6 from janhq/integrateWithJan 

    docs: add instructions for pointing jan fe to the engine
    @0xSage
    0xSage authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    74f22f0 View commit details
    Browse the repository at this point in the history

  18. Add CURL and Invoke-WebRequest to README

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    39dc0b4 View commit details
    Browse the repository at this point in the history

  19. Add llamacorn etc to README

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    bab77ca View commit details
    Browse the repository at this point in the history

  20. Incorporate instructions on how to run on Jan

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    0c57e98 View commit details
    Browse the repository at this point in the history

  21. Merge pull request #7 from janhq/update-readme 

    README: add Quickstart, Model Load and Inference Request docs
    @dan-homebrew
    dan-homebrew authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    089c3c2 View commit details
    Browse the repository at this point in the history

  22. Add Executable Package as Step 1 of Readme

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    3fd4846 View commit details
    Browse the repository at this point in the history

  23. Additional cleanup of README

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    4180286 View commit details
    Browse the repository at this point in the history

  24. Merge pull request #8 from janhq/update-readme 

    Add Package Contents, and README
    @dan-homebrew
    dan-homebrew authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    1be74df View commit details
    Browse the repository at this point in the history

  25. Remove orphaned heading

    @dan-homebrew
    dan-homebrew committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    ebcba19 View commit details
    Browse the repository at this point in the history

  26. Merge pull request #9 from janhq/update-readme 

    Remove orphaned heading
    @dan-homebrew
    dan-homebrew authored Mar 8, 2024
    Configuration menu
    Copy the full SHA
    dcc7fcb View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2024

  1. Set build type

    @tikikun
    tikikun authored Mar 10, 2024
    Configuration menu
    Copy the full SHA
    ef0158b View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. add naive hiding stop words case

    automaticcat committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    984d95d View commit details
    Browse the repository at this point in the history

  2. delete redundant logic and debug log

    automaticcat committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    a5255c8 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #14 from janhq/10-epic-add-proper-handler-for-stop… 

    …-words

Add naive hiding stop words case
    @tikikun
    tikikun authored Mar 11, 2024
    Configuration menu
    Copy the full SHA
    635e6bf View commit details
    Browse the repository at this point in the history

  4. Sync rel and main branch new commit for CICD

    @hiento09 @hientominh
    hiento09 authored and hientominh committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    00c4b79 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #17 from janhq/chore/cherrypick-ci-cd 

    Sync rel and main branch new commit for CICD
    @hiento09
    hiento09 authored Mar 11, 2024
    Configuration menu
    Copy the full SHA
    504058c View commit details
    Browse the repository at this point in the history

  6. remove unuse task in CI (#18) 

    Co-authored-by: Hien To <[email protected]>
    @hiento09 @hientominh
    hiento09 and hientominh committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    e6fe8ae View commit details
    Browse the repository at this point in the history

  7. Merge pull request #19 from janhq/chore/cherrypick-update-ci-cd 

    remove unuse task in CI (#18)
    @hiento09
    hiento09 authored Mar 11, 2024
    Configuration menu
    Copy the full SHA
    849afe2 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. Update links to Binaries and Models

    @dan-homebrew
    dan-homebrew committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    71b671f View commit details
    Browse the repository at this point in the history

  2. Merge pull request #20 from janhq/update-readme 

    Update link to Nitro Executable and LlamaCorn Model Engine
    @dan-homebrew
    dan-homebrew authored Mar 12, 2024
    Configuration menu
    Copy the full SHA
    4bc539d View commit details
    Browse the repository at this point in the history

  3. Add description of LlamaCorn

    @dan-homebrew
    dan-homebrew committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    2776d2d View commit details
    Browse the repository at this point in the history

  4. Merge pull request #22 from janhq/update-readme 

    Add description of LlamaCorn
    @dan-homebrew
    dan-homebrew authored Mar 12, 2024
    Configuration menu
    Copy the full SHA
    6dce26d View commit details
    Browse the repository at this point in the history

  5. fix: correct model links

    @hahuyhoang411
    hahuyhoang411 authored Mar 12, 2024
    Configuration menu
    Copy the full SHA
    4d9ac10 View commit details
    Browse the repository at this point in the history

  6. chore: remove redundant var

    automaticcat committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    854b03e View commit details
    Browse the repository at this point in the history

  7. Merge pull request #23 from janhq/chore/update-readme 

    Chore: Update README
    @hahuyhoang411
    hahuyhoang411 authored Mar 12, 2024
    Configuration menu
    Copy the full SHA
    ff0ac24 View commit details
    Browse the repository at this point in the history

  8. generic fix for chatml and llama model in general

    automaticcat committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    d229a58 View commit details
    Browse the repository at this point in the history

  9. hotfix remove reundant info

    automaticcat committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    da5b54e View commit details
    Browse the repository at this point in the history

  10. Merged upstream changes and resolved conflicts.

    @tikikun
    tikikun committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    85d5aa0 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. fix im_end words filtering

    automaticcat committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    6affd16 View commit details
    Browse the repository at this point in the history

  2. latest

    automaticcat committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    bd51db1 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main'

    automaticcat committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    875da56 View commit details
    Browse the repository at this point in the history

  4. fix missing starting token

    @tikikun
    tikikun committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    9e4c6ab View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. fix for edge case of exiting word

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    e797f60 View commit details
    Browse the repository at this point in the history

  2. remove redundant parts

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    4d0f119 View commit details
    Browse the repository at this point in the history

  3. simpler stop word for efficiency

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    7053920 View commit details
    Browse the repository at this point in the history

  4. remove redundant include

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    9e56ff0 View commit details
    Browse the repository at this point in the history

  5. better default value

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    3b199d4 View commit details
    Browse the repository at this point in the history

  6. proper way to handle prompt template

    @tikikun
    tikikun committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    77488be View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Update tensorrtllm.cc

    @tikikun
    tikikun authored Mar 15, 2024
    Configuration menu
    Copy the full SHA
    49ceb40 View commit details
    Browse the repository at this point in the history

  2. add destroy

    @tikikun
    tikikun committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    a0df72f View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. sync upstream

    @tikikun
    tikikun committed May 6, 2024
    Configuration menu
    Copy the full SHA
    6299d4d View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. upgrade upstream

    @tikikun
    tikikun committed May 13, 2024
    Configuration menu
    Copy the full SHA
    745a87b View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. fix some minor changes in apis and default value

    Alan Dao committed May 15, 2024
    Configuration menu
    Copy the full SHA
    1f2b57f View commit details
    Browse the repository at this point in the history

  2. tutorial

    Alan Dao committed May 15, 2024
    Configuration menu
    Copy the full SHA
    6570d57 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. feat: Init code for cortex.tensorrtllm

    @CameronNg
    CameronNg committed May 22, 2024
    Configuration menu
    Copy the full SHA
    ebd312f View commit details
    Browse the repository at this point in the history

  2. Merge pull request #34 from janhq/feat/init_cortex_tensorrtllm 

    feat: Init code for cortex.tensorrtllm
    @tikikun
    tikikun authored May 22, 2024
    Configuration menu
    Copy the full SHA
    ce536d4 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2024

  1. Draft commit

    @CameronNg
    CameronNg committed May 25, 2024
    Configuration menu
    Copy the full SHA
    12ebcfd View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. feat: Cortex.tensorrtllm - Add example server and e2e test

    @CameronNg
    CameronNg committed May 27, 2024
    Configuration menu
    Copy the full SHA
    9b6e7d9 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. Revert "feat: Init code for cortex.tensorrtllm" (#36)

    @CameronNg
    CameronNg authored May 29, 2024
    Configuration menu
    Copy the full SHA
    9308211 View commit details
    Browse the repository at this point in the history

  2. Merge main

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    8898597 View commit details
    Browse the repository at this point in the history

  3. Rename nitro to cortex.tensorrtllm

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    2fcadb9 View commit details
    Browse the repository at this point in the history

  4. fix: rename tensorrtllm to tensorrt-llm

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    f322e65 View commit details
    Browse the repository at this point in the history

  5. test:Test quality-gate file

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    5504e49 View commit details
    Browse the repository at this point in the history

  6. test:Test quality-gate file v2

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    36ee716 View commit details
    Browse the repository at this point in the history

  7. test:Test quality-gate file v3

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    fdc79f7 View commit details
    Browse the repository at this point in the history

  8. test:Test quality-gate file v4

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    9c83bd3 View commit details
    Browse the repository at this point in the history

  9. test:Test quality-gate file v5

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    5530de2 View commit details
    Browse the repository at this point in the history

  10. test:Test quality-gate file v6

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    69e3c12 View commit details
    Browse the repository at this point in the history

  11. test:Test quality-gate file v7

    @CameronNg
    CameronNg committed May 29, 2024
    Configuration menu
    Copy the full SHA
    a66f2ba View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. test:Test quality-gate file v8

    @CameronNg
    CameronNg committed May 30, 2024
    Configuration menu
    Copy the full SHA
    acb3728 View commit details
    Browse the repository at this point in the history

  2. test:Test quality-gate file v9

    @CameronNg
    CameronNg committed May 30, 2024
    Configuration menu
    Copy the full SHA
    83310d0 View commit details
    Browse the repository at this point in the history

  3. test:Test quality-gate file v10

    @CameronNg
    CameronNg committed May 30, 2024
    Configuration menu
    Copy the full SHA
    16b317a View commit details
    Browse the repository at this point in the history