Citation Tracking Back All the Way to the Original Text Chunk?

[CHENGLING-QIU]

Hi team,
Question: Is there an existing solution to trace back the citation all the way to the original text?

More background:
I would like to trace back from the citation, e.g. Repots(ID1, ID2), Entities(ID1, ID2) all the way to the original text chunks; so that I can easily make sure if the generation is actual relevant. This is critical because even just looking at the traced report or entity level, I would have to worry about if these contents are hallucinated.

I didn't find discussion on that after some research (the most relevant ask seems to be issue_729

To achieve that, I am looking into the response object from the search. Looking into result.context_data & result.context_text, there are actually the contents for these context.

However:

1. It seems all the items (definitely more than these in the response) are returned. Ideally, I really just want these got cited. I am not sure if these cited types and corresponding IDs are included from the search result somehow?
   e.g.
   result.context_data["reports"] returns all the reports seemingly, not just these cited in this particular search
   Context data:

      id  ... rank
0    284  ...  9.0
1    385  ...  8.5
2    271  ...  8.5
3    381  ...  8.5
4    322  ...  8.5
..   ...  ...  ...
343  318  ...  8.5
344  169  ...  8.5
345  159  ...  7.5
346   33  ...  8.5
347  440  ...  8.5

2. Again, it seems there is no way to directly trace back to the original text chunk before any GraphRAG processing. Even the context_text contains the internal content that has the continuous citation to other level.

It would be nice if the functionality tracing back to the original text is already built in, and if so, what's the best to retrieve that?
If not, it will still be great if you could give me some suggestions how to implement given a search response.

Thank you!

[KylinMountain]

i have implemented please check out my fork of graph tag.

[CHENGLING-QIU]

i have implemented please check out my fork of graph tag.

@KylinMountain Thank you for sharing that! I am looking into your commits, it seems like the most relevant update is these get_index_data methods you implemented. Just want to confirm that!
webserver/search/indexdata.py
14f24b8#diff-0993d32b8e1fe7190cf34482a2164a18e677224b487345b9fe465b6346080f56

Also, it will be great if you can share the bootstrap instructions for your fork so that we can try your updates easier!

[KylinMountain]

I have updated the README

[CHENGLING-QIU]

I have updated the README

@KylinMountain Thank you! I was able to host your improved version and some minor changes to redirect the references. It works well and I am just curious is there a chat UI built? Since I believe you have some streaming output optimized

[KylinMountain]

you can download the desktop app at here https://cherry-ai.com/

[lawyinking]

It seems like we can only trace back to the original text when the citation is 'source' (which actually refers to 'create_base_text_units.parquet').

For answers with citations that are not 'source', but 'relations', 'communities', 'entities', etc., I believe it is impossible to trace back to the original text. These are summarized through procedures during the construction of the graph by the LLM. Ultimately, the traceback would consist of a series of 'text_unit_ids' (that's why it has coherence in RAG, it is a graph constructed by LLM, it answering us using the 'relationship' info) used to summarize those entities, communities, etc., as shown in the example below in the .parquet file:

['0212e3baf878b6ad7b7e2f4ba289104a', '04b7c2cab0cc1098d8606cec07f7c8d5', '087fb7fb40330a1589f876a93ee83dde', ...]

to achieve a better accuracy of RAG, best way today should be Graph, but graph has this limitation.

[lawyinking]

If you prefer GraphRAG to answer based only on 'source' sometimes, you can type something like '... answer with reference only to data sources, but not to entities, relationships, or communities' in your prompt, ofc It's only for local search (if you prefer not 'too global'), and it's meaningless for global search.