entity graph seems not related to input data

[hyeoungho]

Hello,

I am trying to run the 'Get Started' example using local LLM (llama 3.1 8b).

Here is my setup:
LLM: hosted by vllm.openai.api_server
Embedding: hosted by open.text.embeddings

settings.yaml:
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: meta-llama/Meta-Llama-3.1-8B
model_supports_json: true # recommended if this is available for your model.
request_timeout: 1000.0
api_base: http://localhost:5000/v1/

embeddings:
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: text-embedding-3-small
api_base: http://localhost:5050/v1/

I applied the changes that namin mentioned: #321

The issue is local search example works fine but global search doesn't seem to be correct. Then I looked at the entity generation step. The names mentioned in the progress step is not relevant to the input text ("A Christmas Carol").

Here is the snippet of the entity generation:
<graphml xmlns="http://graphml.graphdrawing.or...
 create_base_entity_graph
level clustered_graph
0 0 <graphml xmlns="http://graphml.graphdrawing.or...
/mnt/e/Work/GraphRag/.venv/lib/python3.11/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
return bound(*args, **kwds)
/mnt/e/Work/GraphRag/.venv/lib/python3.11/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
return bound(*args, **kwds)
 create_final_entities
id name type ... graph_embedding text_unit_ids description_embedding
0 b45241d70f0e43fca764df95b2b81f77 "TAYLOR" "PERSON" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.009118832647800446, -0.05749783664941788, 0...
1 4119fd06010c494caa07f439b333f4c5 "JORDAN" "PERSON" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.026424549520015717, -0.047401491552591324, ...
2 d3835bf3dda84ead99deadbeac5d0d7d "CRUZ" "PERSON" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [-0.0002833358012139797, -0.04008307680487633,...
3 077d2820ae1845bcbb1803379a3d1eae "THE DEVICE" "TECHNOLOGY" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.027066845446825027, -0.04150278866291046, 0...
4 3671ea0dd4e84c1a9b02c5ab2c8f4bac "ALEX" NaN ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.036357663571834564, -0.046841856092214584, ...
5 19a7f254a5d64566ab5cc15472df02de "OPERATION: DULCE" "MISSION" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.0423923060297966, -0.045900922268629074, 0....
6 e7ffaee9d31d4d3c96e04f911d0a8f9e "THE TEAM" "ORGANIZATION" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.036179106682538986, -0.04838939011096954, 0...
7 f7e11b0e297a44a896dc67928368f600 "WASHINGTON" NaN ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.04118547588586807, -0.06376904994249344, 0....
8 1fd3fa8bb5a2408790042ab9573779ee "CONTROL" "CONCEPT" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.027394650503993034, -0.029524872079491615, ...
9 27f9fbe6ad8c4a8b9acee0d3596ed57c "INTELLIGENCE" "CONCEPT" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.019726935774087906, -0.05124255642294884, 0...
10 e1fd0e904a53409aada44442f23a51cb "FIRST CONTACT" "EVENT" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.056874439120292664, -0.06274805217981339, 0...
11 de988724cfdf45cebfba3b13c43ceede "HUMANITY'S RESPONSE" "EVENT" ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.04697030782699585, -0.045056700706481934, 0...
12 96aad7cb4b7d40e9b7e13b94a67af206 "SAM RIVERA" NaN ... None [10730234d6ccc7cee08f3cfc58d8a9a1, 999c9887098... [0.03408035263419151, -0.06179048866033554, 0....
13 c9632a35146940c2a86167c7726d35e9 "CRUZ"<<|IM_START|>USER-GOAL-GIVEN A TEXT DOCU... NaN ... None [10730234d6ccc7cee08f3cfc58d8a9a1] [0.025774933397769928, -0.042845964431762695, ...
14 9646481f66ce4fd2b08c2eddda42fc82 "CRUZ"<|<|IM_START|>USER-GOAL-GIVEN A TEXT DOC... NaN ... None [999c9887098d1a25dc3b42a8da7ddc8c, f40e4b274b5... [0.024699920788407326, -0.042923908680677414, ...

I don't know where those entity names were from. I guess that might be the reason of my global search failure. Any suggestions?

Thank you.

[natoverse]

Those entities look like they come from the Operation: Dulce synthetic test story we have bundled in the repo

[hyeoungho]

Thanks Nathan for the answer. Do you happen to know in what circumstances the test story comes into play while generating entity graph?