New Mode that specifically references all uploaded/local files && Bulk Ingestion of a Documents/Files Folder

[K-J-VV]

Currently it seems like the only way to have the LLMs "speak" about documents/files is to manually select the ones I want information on.

There's a project called Private GPT (https://github.com/zylon-ai/private-gpt), it's focus is on allowing you to ask questions about your documents using LLMs.

Would it be possible to create a similar 'Documents/Query' mode? I'd really like to be able to have a mode where the LLM I choose is specifically querying all the local files I have uploaded to LibreChat. Further, it'd be great if this Documents/Query mode were added that there is a folder that could me mounted to the container that is automatically ingested. So when a container is spun up, the RAG api automatically begins parsing/searching through the documents & monitors for any changes to the folder in case more files are added to this folder

[danny-avila]

There will be a way to cluster files together, as if organizing them into folders, with no limit other than the disk space! Also being able to point a file to multiple "clusters", so to speak. Still drafting what this might look like from the interface.

Monitoring files is also a good idea, that could be an addition to this on top of the new "clustering" of files

[K-J-VV]

Also being able to point a file to multiple "clusters", so to speak. Still drafting what this might look like from the interface.

This is a great idea. A practical use case: different groups of users may only need access to a specific cluster for organization/security/relevance

[krahnikblis]

came to log a similar capability enhancement - i'm using this in a corporate setting as a UI for my own team and the parent group - would really like to have a shared library of files instead of just users' personally loaded files. would be cool to divide them into folders or buckets (so we on the data team can put a bunch of techy stuff in a folder, and the larger group could look at that if they want but they won't, their folders would have powerpoints, PDFs, docs, etc.).

in parallel, maybe i need to open or comment another topic, but would love to have group chat capabilities, so that for example my team could all share a single chat wherein we do our dev stuff collaboratively, and can reference prior prompts/answers. and finally, ability to prune a thread history - such as when we ask for code, the code is wrong, we iterate and fix, would like to delete the wrong answer from the thread so we only have good info in conversation...

[danny-avila]

great comments, I'm excited to share there is a lot of work started on your first few points already

finally, ability to prune a thread history - such as when we ask for code, the code is wrong, we iterate and fix, would like to delete the wrong answer from the thread so we only have good info in conversation...

This is already possible, and I do it all the time with LibreChat, see this comment for more info:

#2387 (comment)

[m49808]

+1 to the first points here. Specifically the items around a shared corporate repository of documents available to all users. Bonus points if those corporate files are "pre-loaded" as an option so the user can surface this info without having to click the docs every time. This would be a huge win for us.

[ascheman]

Currently it seems like the only way to have the LLMs "speak" about documents/files is to manually select the ones I want information on.

There's a project called Private GPT (https://github.com/zylon-ai/private-gpt), it's focus is on allowing you to ask questions about your documents using LLMs.

Would it be possible to create a similar 'Documents/Query' mode? I'd really like to be able to have a mode where the LLM I choose is specifically querying all the local files I have uploaded to LibreChat. Further, it'd be great if this Documents/Query mode were added that there is a folder that could me mounted to the container that is automatically ingested. So when a container is spun up, the RAG api automatically begins parsing/searching through the documents & monitors for any changes to the folder in case more files are added to this folder

I like this idea, thanks for bringing it up, as I have a similar requirement. Besides providing a complete solution, I could imagine a more modularized approach. If the rag_api would be extended to upload documents, this would enable several solutions:

• one could implement their own document loader,
• there could be a general one in the long run which works in the required way by automatically indexing/uploading documents in a particular configurable) directory.

In my context I have particular (HTML) documents which contain only a small relevant payload while having much boilerplate (CSS/JS/Navigation/...) which could be filtered out from the input, making it much smaller (and thereby way faster to parse and load).