Input time and date dynamically in modelSpec

[mrn-linak]

Hi all,

In our current chat implementation we use a dynamic system message which includes todays date. We have found that this helps ground the model and reduce hallucination, when it comes to questions about events, places or people.

SYSTEM_MESSAGE = f"""You are an AI Assistant. Your knowledge cutoff is October, 2023. Today is {current_date}"""

Possible solution

My immediate thought was to rebuild our app every day at midnight, and simple inject the date into the librechat.yml file. It isn't very elegant, but it would do the job. But:

• Are there any other ways to achieve this?
• Does modelSpec support system message?

If there are no other ways to support this, this can be considered a feature request.

[dareima]

We would also very much like to have that. Anthropic and OpenAI are doing this for ChatGPT and Claude as well, btw.

[RW21]

As @dareima mentioned, Anthropic does this in their own system prompt:

<claude_info> The assistant is Claude, created by Anthropic. The current date is {}.

https://docs.anthropic.com/en/release-notes/system-prompts#july-12th-2024

It would be great to have a dynamic templating system for these kind of prompts.

[owengo]

As far as I understand the code, currently the only evaluation of the "current user" and "current side" occurs client-side and only the "prompt system" where you can configure static prompt.

A possible solution would be to implement the {current_datetime} replacement server side, maybe a the client level when the "promptPrefix" is evaluated. We could generate a full datetime with timezone ( we don't know the time zone client side ).

[dareima]

@owengo would querying an NTP server be another possible solution attempt?

Via the .env file something like:

NTP_SERVER=192.53.103.108
NTP_POLLING_INTERVAL_MS=5000
NTP_LOCAL_TIMEZONE=GMT+2

[owengo]

The problem is not to know what time it is: If this is done client side we have the time from the browser, if this is done server side we have the time of the server ( which hopefully is synchronised via ntp or other protocols )
the problem is to inject the information in the prompt sent to the llm.

[dareima]

@owengo understood.
Then I suppose finding a way to inject it with a variable in system prompts / prompt prefixes, like OpenAI and Anthropic are doing it, will be the most practical way to do this.

[owengo]

I have just submitted: #4335 and LibreChat-AI/librechat.ai#132 , which allow to use {{current_date}} in presets for OpenAI, Anthropic and Assistants. It's probably straigthforward to do the same for other models but I can't test them right now.

[owengo]

I have submitted #4603 , which has benefits:

• now {{current_date}} and {{current_user}} are replaced from instructions as for custom prompts
• the evaluation is 100% client-side
• the pr is on a dedicated branch, I messed up the former one.

[AIndoria]

Does this also work for Custom Instructions rather than just prompts? Because I modified the original file with your changes, added those placeholders to custom instructions for the model, and it doesn't recognize it.

[owengo]

This works with the "additionnal instructions" from the settings, not the "custom instructions" of the model.

[AIndoria]

hm, good to know - Any way to make it work with the custom instructions of the model itself?

…

On Mon, Nov 11, 2024 at 1:00 AM owengo ***@***.***> wrote: This works with the "additionnal instructions" from the settings, not the "custom instructions" of the model. — Reply to this email directly, view it on GitHub <#3713 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACEE27GBDBQMN67TFBMFG4T2ABP2BAVCNFSM6AAAAABMZ3GRPOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRRGE3DOOI> . You are receiving this because you commented.Message ID: ***@***.***>

[owengo]

It's technically possible to override the instructions for each run:
https://platform.openai.com/docs/api-reference/runs/createRun

And it would probably be possible to modify my patch to apply it also on the instructions, but I would like to have this PR accepted before.

If you want to test it yourself just modify:

    // Check and replace special variables in promptPrefix
    if (conversation?.promptPrefix) {
      conversation.promptPrefix = replaceSpecialVars({
        text: conversation.promptPrefix,
        user
      });
    }
   // add this to replace the variables in the instructions
    if (conversation?.instructions) {
      conversation.instructions= replaceSpecialVars({
        text: conversation.instructions,
        user
      });
    }
    

No guarantee if this will work!