[Bug]: <title>Error embedding chunk {'OpenAIEmbedding': "'NoneType' object is not iterable"} #528
Comments
hongyispace
added
bug
|
this is a temp hacked solution for ollama |
Thank you for replaying. I succeed in finishing the pipeline, but I am looking for solution for the bug when query with local models. |
|
Hi @hongyispace Did you find a solution to fix the bug when local query with local model? |
|
Hi I have hacked it via changing the code in graphrag/query/llm/oai/openai.python "chunck_text" method: (remove token encode, because the local llm just can embed text instead of encoded ones) def chunk_text( |
I tried to change "raphrag/query/llm/text_utils.py" wihch contain "chunk_text". New error is as follows: |
|
The local search with embeddings from Ollama now works. |
|
Consolidating alternate model issues here: #657 |
natoverse
added
community_support
same issue encountered. |
Describe the bug
I have finished the pipeline with llm and ebedding of ollama. However, when I tried to query: python -m graphrag.query --root pg18v8 --method local "describe the types of impact craters"
The error message is:
INFO: Reading settings from pg18v8/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=56', 'type': "openai_chat", 'model': 'llama3:70b-instruct-q5_K_M', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 1, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 1}
creating embedding llm client with {'api_key': 'REDACTED,len=56', 'type': "openai_embedding", 'model': 'nomic-embed-text', 'max_tokens': 4000, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/api', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 3, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error embedding chunk {'OpenAIEmbedding': "'NoneType' object is not iterable"}
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/main.py", line 75, in
run_local_search(
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/cli.py", line 154, in run_local_search
result = search_engine.search(query=query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/structured_search/local_search/search.py", line 118, in search
context_text, context_records = self.context_builder.build_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/structured_search/local_search/mixed_context.py", line 139, in build_context
selected_entities = map_query_to_entities(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/context_builder/entity_extraction.py", line 55, in map_query_to_entities
search_results = text_embedding_vectorstore.similarity_search_by_text(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/vector_stores/lancedb.py", line 118, in similarity_search_by_text
query_embedding = text_embedder(text)
^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/context_builder/entity_extraction.py", line 57, in
text_embedder=lambda t: text_embedder.embed(t),
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/graphrag/query/llm/oai/embedding.py", line 96, in embed
chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/grag/lib/python3.12/site-packages/numpy/lib/function_base.py", line 550, in average
raise ZeroDivisionError(
ZeroDivisionError: Weights sum to zero, can't be normalized
The environmental variables are:
export GRAPHRAG_LLM_MODEL="llama3:70b-instruct"
export GRAPHRAG_EMBEDDING_MODEL="nomic-embed-text"
export GRAPHRAG_LLM_API_BASE="http://localhost:11434/v1"
export GRAPHRAG_EMBEDDING_API_BASE="http://localhost:11434/api"
Any suggestion is appreciated!
Steps to reproduce
No response
Expected Behavior
No response
GraphRAG Config Used
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: llama3:70b-instruct-q5_K_M #可以换成其他模型
model_supports_json: true # recommended if this is available for your model.
max_tokens: 4000
request_timeout: 180.0
api_base: http://localhost:11434/v1
api_version: 2024-02-15-preview
organization: <organization_id>
deployment_name: <azure_model_deployment_name>
tokens_per_minute: 150_000 # set a leaky bucket throttle
requests_per_minute: 10_000 # set a leaky bucket throttle
max_retries: 1
max_retry_wait: 10.0
sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
concurrent_requests: 1 # the number of parallel inflight requests that may be made
parallelization:
stagger: 0.3
num_threads: 50 # the number of threads to use for parallel processing
async_mode: threaded # or asyncio
embeddings:
parallelization: override the global parallelization settings for embeddings
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: nomic-embed-text #号称现在最厉害的embedding模型
api_base: http://localhost:11434/api
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
max_retries: 3
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
#concurrent_requests: 1 # the number of parallel inflight requests that may be made
batch_size: 4 # the number of documents to send in a single request
batch_max_tokens: 2048 # the maximum number of tokens to send in a single request
# target: required # or optional
chunks:
size: 4000
overlap: 500
group_by_columns: [id] # by default, we don't allow chunks to cross documents
input:
type: file # or blob
file_type: text # or csv
base_dir: "input"
file_encoding: utf-8
file_pattern: ".*\.txt$"
cache:
type: file # or blob
base_dir: "cache"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
storage:
type: file # or blob
base_dir: "output/${timestamp}/artifacts"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
reporting:
type: file # or console, blob
base_dir: "output/${timestamp}/reports"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
entity_extraction:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/entity_extraction.txt"
entity_types: [organization,person,geo,event]
max_gleanings: 0
summarize_descriptions:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/summarize_descriptions.txt"
max_length: 500
claim_extraction:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
enabled: true
prompt: "prompts/claim_extraction.txt"
description: "Any claims or facts that could be relevant to information discovery."
max_gleanings: 0
community_report:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/community_report.txt"
max_length: 2000
max_input_length: 8000
cluster_graph:
max_cluster_size: 5
embed_graph:
enabled: false # if true, will generate node2vec embeddings for nodes
num_walks: 10
walk_length: 40
window_size: 2
iterations: 3
random_seed: 597832
umap:
enabled: false # if true, will generate UMAP embeddings for nodes
snapshots:
graphml: true
raw_entities: false
top_level_nodes: false
local_search:
text_unit_prop: 0.5
community_prop: 0.1
conversation_history_max_turns: 5
top_k_mapped_entities: 10
top_k_relationships: 10
max_tokens: 12000
global_search:
max_tokens: 12000
data_max_tokens: 12000
map_max_tokens: 1000
reduce_max_tokens: 2000
concurrency: 32
Logs and screenshots
No response
Additional Information
The text was updated successfully, but these errors were encountered: