Skip to content

Fixes for rwkv-world template and the missing inputs.use_jinja in llama-cli #14336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 23, 2025

Conversation

MollySophia
Copy link
Collaborator

  • Handle the rwkv-world chat template better
  • Fix a bug where inputs.use_jinja is not set according to g_params->use_jinja in llama-cli

@MollySophia MollySophia requested a review from CISC June 23, 2025 03:34
Copy link
Collaborator

@CISC CISC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea if this is the correct way to set the system prompt for RWKV7, but I'm sure you do, so LGTM. :)

Nice catch on use_jinja!

@MollySophia
Copy link
Collaborator Author

I have no idea if this is the correct way to set the system prompt for RWKV7, but I'm sure you do, so LGTM. :)

Thanks for the reminder! I just checked it again with @BlinkDL and it should indeed be another one.

@MollySophia MollySophia merged commit 72c6bc3 into ggml-org:master Jun 23, 2025
47 checks passed
@zhouwg
Copy link
Contributor

zhouwg commented Jun 23, 2025

@MollySophia, nice to meet you and sorry to bother you.

today I found your excellent project https://github.com/MollySophia/rwkv-qualcomm
and read the following file carefully.
https://github.com/MollySophia/rwkv-mobile/blob/master/src/backends/qnn/qnn_backend.cpp

as you said in your rwkv-qualcomm project:
MollySophia/rwkv-qualcomm#4 (comment)

I totally agree with what you said: the approach in your project is exactly similar to the approach in the official QNN solution, or it's also an End-to-End inference solution.

obviously, you are an AI expert and also good at/familiar with Qualcomm's QNN SDK.

I know very very very very little knowledge about hardcore AI tech although I'm good at Android sys dev and dev in the embedded system.

I have a candidate PR in the llama.cpp project:#12326

could you help to take a look if you have time?

thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants