Custom prompt templates for models

[darinchau]

I was wondering if there are ways to pass in a custom prompt template for the LLMs; some LLMs would greatly benefit from a specified prompt template. For example, Llama 2 Instruct with the [INST]<prompt>[/INST] template.

Would be happy to open a PR if needed. Thanks!

[Wintoplay]

Agree, we need to fulfill this instruction/chat template for each model

[joaomdmoura]

Yup! we layed down the work for this in this version, so it's coming in the next one, pretty soon, sorry it took soo long

[kam-st]

Yup! we layed down the work for this in this version, so it's coming in the next one, pretty soon, sorry it took soo long

This would be great, as it is really frustrating when agents can formulate Action and Action Input properly. Especially with Open LLM.

Or do you think templated won't solve this problem?

[MarcoLooy]

This post is 2 months old. Is there an update?

[gadgethome]

If you are on the latest release, you can try this. Also, consider joining the discord channel as there is a lot of good material on here https://discordapp.com/channels/1192246288507474000/1235578548924977173/1235583184108916821

Agent = Agent(
role="{topic} specialist",
goal="Figure {goal} out",
backstory="I am the master of {role}",
system_template="""<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>""",
prompt_template="""<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>""",
response_template="""<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>""",
)

[MarcoLooy]

@gadgethome, super cool. Thanks!