Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move beam search in case of chat scenario to sampler.cpp #1215

Merged
merged 2 commits into from
Dec 21, 2024
Merged

Move beam search in case of chat scenario to sampler.cpp #1215

merged 2 commits into from
Dec 21, 2024

Conversation

sbalandi
Copy link
Contributor

@sbalandi sbalandi commented Nov 14, 2024 •
edited
Loading

Task CVS-156578

  • add missed token, if prev generation was finished because max length was reached

@github-actions github-actions bot added category: visual language Visual language pipeline category: continuous batching Continuous batching category: LLM LLM pipeline (stateful, static) category: sampling Sampling / Decoding algorithms labels Nov 14, 2024
@sbalandi sbalandi marked this pull request as ready for review November 14, 2024 18:11
@sbalandi sbalandi requested a review from Wovchena November 14, 2024 18:11
@ilya-lavrenov ilya-lavrenov added this to the 2025.0 milestone Nov 15, 2024
Wovchena
Wovchena requested changes Nov 15, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/lm_encoding.cpp Outdated Show resolved Hide resolved
src/cpp/src/lm_encoding.cpp Outdated Show resolved Hide resolved
src/cpp/src/lm_encoding.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Nov 15, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/utils.cpp Outdated Show resolved Hide resolved
src/cpp/src/visual_language/pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
@ilya-lavrenov ilya-lavrenov mentioned this pull request Nov 15, 2024
Merged
@github-actions github-actions bot added no-match-files and removed category: sampling Sampling / Decoding algorithms labels Nov 20, 2024
@sbalandi
Copy link
Contributor Author

the tests are passed, but it still doesn't work for bigger max_token size for TinyLlama-1.1B-Chat-v1.0 . Tokenizer converts first token from symbol to _symbol. So please, do not review.

ilya-lavrenov
ilya-lavrenov reviewed Nov 21, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
@sbalandi sbalandi force-pushed the beam branch 3 times, most recently from f68e9db to 6f9335e Compare November 25, 2024 10:41
@sbalandi sbalandi mentioned this pull request Dec 13, 2024
Merged
@sbalandi sbalandi force-pushed the beam branch 3 times, most recently from 66cd315 to ab3bcdc Compare December 16, 2024 18:47
@sbalandi sbalandi mentioned this pull request Dec 16, 2024
Merged
@sbalandi sbalandi force-pushed the beam branch 2 times, most recently from a5d41fa to e2752c1 Compare December 19, 2024 12:29
Wovchena
Wovchena approved these changes Dec 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/utils.hpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/utils.hpp Outdated Show resolved Hide resolved
src/cpp/src/visual_language/inputs_embedder.hpp Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Dec 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/visual_language/inputs_embedder.cpp Outdated Show resolved Hide resolved
src/cpp/src/visual_language/inputs_embedder.cpp Outdated Show resolved Hide resolved
src/cpp/src/visual_language/inputs_embedder.cpp Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the category: continuous batching Continuous batching label Dec 20, 2024
@sbalandi sbalandi force-pushed the beam branch 2 times, most recently from 0ce4c46 to fa36a48 Compare December 20, 2024 15:17
@ilya-lavrenov ilya-lavrenov added this pull request to the merge queue Dec 20, 2024
Merged via the queue into openvinotoolkit:master with commit 05d01ac Dec 21, 2024
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Wovchena Wovchena Wovchena approved these changes

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

Assignees

@ilya-lavrenov ilya-lavrenov

@Wovchena Wovchena

Labels
category: LLM LLM pipeline (stateful, static) category: visual language Visual language pipeline no-match-files
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
3 participants
@sbalandi @ilya-lavrenov @Wovchena