Add Chat history endpoints

[yangcao77]

Description

Add Chat history endpoints

GET /conversations
List all conversation_id for a login user and also the summary of each conversation.

GET /conversations/:conversation_id
Get chat history in the format of Langchain message for the requested conversation_id

DELETE /conversations/:conversation_id:
Get the conversation in cache for the requested conversation_id

Please check this screen recording for a quick demo: https://drive.google.com/file/d/1UFSq8W6BzRcaELiZuDYbjtUxzNOLxxU0/view?usp=sharing

in addition, update the conversation cache to use langchain message type instead of plain text for future enhance. i.e. image in humanMessage or AIMessage can be added as part of the object.content.

AIMessage({
    content: [{
            type: 'text',
            text: response,
        },{
            type: 'image_url',
            image_url: fileBase64,
        },
    ],
});

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change

Related Tickets & Documents

• Related Issue [RFE]Use Langchain standard message types for storing chat cache #202
  https://issues.redhat.com/browse/RHDHPAI-175
  https://issues.redhat.com/browse/RHDHPAI-193

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[tisnik]

It looks good in overall, will have some nitpicks. Are you going to squash commits first please?

Also - would it be possible please to have one PR that changes just transcript format. IIRC I have created the transcript history based on HumanMessage and AIMessage sequence, but it was later changed to plaintext. @asamal4 do you recall why?

[asamal4]

Also - would it be possible please to have one PR that changes just transcript format. IIRC I have created the transcript history based on HumanMessage and AIMessage sequence, but it was later changed to plaintext. @asamal4 do you recall why?

@tisnik
Primary reason was to remove unnecessary dependency on langchain. There were few discussions regarding whether we continue to use langchain or not. So while storing history we wanted to use native python object and modify accordingly during run time.
We just needed a mechanism to identify/differentiate user query vs ai response, whether it was langchain message object or python dict or string with special characters didn't matter to us then.

[yangcao77]

It looks good in overall, will have some nitpicks. Are you going to squash commits first please?

When PR merges, github will have a option to squash all commits into one. so I will leave this work to be handled by PR merge.

Primary reason was to remove unnecessary dependency on langchain. There were few discussions regarding whether we
continue to use langchain or not. So while storing history we wanted to use native python object and modify accordingly during run time.
We just needed a mechanism to identify/differentiate user query vs ai response, whether it was langchain message object or python dict or string with special characters didn't matter to us then.

restricting plain text in the cache comes with significant limitations:

1. Loss of information
   Storing plain text strips away metadata and information that consumers may need. For instance, timestamps, model names, server identifiers, and other key details we would like to also save within messages are essential to for UI to provide a richer and more complete message history.

2. Limited Scalability for Future Enhancements
   For any potential future functions for more complicated cases, for example non-text query and response, leveraging frameworks like LangChain offers a more robust and scalable solution, with mechanisms explicitly designed to handle these complex scenarios.

3. Interoperability and Flexibility
   road-core service aims to be a generic backend solution for multiple Lightspeed teams.
   For RHDH Lightspeed, where Node.js is used instead of Python, directly importing road-core as a library isn’t feasible. Instead, we rely on image containers. To ensure seamless compatibility, it's preferable to adopt a standardized public object structure. LangChain’s message schema, which is consistently designed across both Python and Node.js, is far more reliable than using a custom-defined object that may frequently change.

[asamal4]

restricting plain text in the cache comes with significant limitations:

1. Loss of information
   Storing plain text strips away metadata and information that consumers may need. For instance, timestamps, model names, server identifiers, and other key details we would like to also save within messages are essential to for UI to provide a richer and more complete message history.
2. Limited Scalability for Future Enhancements
   For any potential future functions for more complicated cases, for example non-text query and response, leveraging frameworks like LangChain offers a more robust and scalable solution, with mechanisms explicitly designed to handle these complex scenarios.
3. Interoperability and Flexibility
   road-core service aims to be a generic backend solution for multiple Lightspeed teams.
   For RHDH Lightspeed, where Node.js is used instead of Python, directly importing road-core as a library isn’t feasible. Instead, we rely on image containers. To ensure seamless compatibility, it's preferable to adopt a standardized public object structure. LangChain’s message schema, which is consistently designed across both Python and Node.js, is far more reliable than using a custom-defined object that may frequently change.

@yangcao77 Agree with you regarding plain text issue. I was just saying why we used plain text, requirement was very simple for us earlier. Not saying that we need to continue using plain text.

However I do have concern about using langchain message object. There are still some discussion not to use langchain for llm call. If we use something else, then we still have to use langchain only for message object. Having langchain dependency only for storing messages doesn't seem right to me. cc: @tisnik

[yangcao77]

However I do have concern about using langchain message object. There are still some discussion not to use langchain for llm call. If we use something else, then we still have to use langchain only for message object. Having langchain dependency only for storing messages doesn't seem right to me.

Re: message type for llm call. with Langchain Object, message content can be easily extract out via message.content
for cases need to use non langchain object, i.e. for Granite models, currently it's still converted use Granite preferred format str

service/ols/src/prompts/prompt_generator.py

Lines 43 to 49 in 97ded98

	new_message = copy(message)
	# Granite specific formatting for history
	if isinstance(message, HumanMessage):
	new_message.content = "\n<|user|>\n" + message.content
	else:
	new_message.content = "\n<|assistant|>\n" + message.content
	return new_message

service/ols/src/prompts/prompt_generator.py

Lines 114 to 115 in 97ded98

	for message in self._history:
	llm_input_values["chat_history"] += message.content

[asamal4]

However I do have concern about using langchain message object. There are still some discussion not to use langchain for llm call. If we use something else, then we still have to use langchain only for message object. Having langchain dependency only for storing messages doesn't seem right to me.

Re: message type for llm call. with Langchain Object, message content can be easily extract out via message.content for cases need to use non langchain object, i.e. for Granite models, currently it's still converted use Granite preferred format str

service/ols/src/prompts/prompt_generator.py

Lines 43 to 49 in 97ded98

	new_message = copy(message)
	# Granite specific formatting for history
	if isinstance(message, HumanMessage):
	new_message.content = "\n<|user|>\n" + message.content
	else:
	new_message.content = "\n<|assistant|>\n" + message.content
	return new_message

service/ols/src/prompts/prompt_generator.py

Lines 114 to 115 in 97ded98

	for message in self._history:
	llm_input_values["chat_history"] += message.content

I am talking about not to use langchain message object while storing. We can use python native dict. Just to avoid any further code related to langchain.

Langchains AIMessage, HumanMessage roughly means {"role": "assistant", "content": "Model Response"}, {"role": "user", "content": "User query"} (this is common for openai). But langchain object uses type instead of role.
If we move to some other package (example litellm), then we don't have to use AIMessage, HumanMessage. we can directly use something similar to above dict object.
As we are changing how we store message now. We can simply use a key-value pair object to store history. This way in future (if we don't use langchain package) we don't have to change how we store again (and no impact on how we retrieve).

But I don't want to create any confusion. Currently it is okay to use langchain objects. Because removing langchain will be a big effort. so I guess we can also change how we store the messages at that time or write logic to recreate similar structure.

[tisnik]

@yangcao77 so if @asamal4 is ok with introducing langchain-based message classes, we might merge right? Don't worry about Ruff, that's my problem to fix it later ;)