Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[.NET] Phi-3 sample response is the same as input prompt #551

Closed
luisquintanilla opened this issue May 31, 2024 · 2 comments · Fixed by #553
Closed

[.NET] Phi-3 sample response is the same as input prompt #551

luisquintanilla opened this issue May 31, 2024 · 2 comments · Fixed by #553

Comments

@luisquintanilla
Copy link
Member

luisquintanilla commented May 31, 2024

When running the HelloPhi sample using Phi-3, the generated response is the same as the user input.

Example Output:

-------------
Hello, Phi!
-------------
Please enter option number:
1. Complete Output
2. Streaming Output
1
Prompt:
What is AI?
Output:
What is AI?

Prompt:
What is ML? 
Output:
What is ML?
Prompt:

Additional Info:

model: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4
OS: Windows 11
ORT GenAI Version: 0.3.0-rc2
sample: https://github.com/microsoft/onnxruntime-genai/tree/main/examples/csharp/HelloPhi
hardware: CPU
.NET version: 6.0

@baijumeswani
Copy link
Contributor

Would you please try adding the chat template to your prompt?

<|user|>What is AI?<|end|><|assistant|>

@luisquintanilla
Copy link
Member Author

@baijumeswani that fixed it. PR #553 ready when you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants