Mac OS File Limit Causing Errors on Verb create_base_entity_graph

[tpsmith130]

Hello all. Wanted to share, as this stumped me for a few hours. If you get the below error over and over, followed by LLM errors, it's not an LLM error or limit being reached! Change your ulimit from 256 to something higher. This is not a persistent change and only applies to the shell you run it in. I set mine to 10240 "ulimit -n 10240" and no longer am seeing errors on Verb create_base_entity_graph. It hasn't finished yet, but sharing just as well.

P.S. if this is in the wrong place, apologies!

OSError: [Errno 24] Too many open files: '/Users/REDACTED/Documents/ragtest/cache/entity_extraction/chat_3e67856d62906f59d2fbe6782471fc47116b854b480de7e2753b81a16e08946f_v2'
13:22:50,979 graphrag.callbacks.file_workflow_callbacks INFO Entity Extraction Error details={'doc_index': 0, 'text': 'REDACTED'}
13:22:51,6 graphrag.callbacks.file_workflow_callbacks INFO Error Invoking LLM details={'prompt': 'REDACTED' {}}
13:22:51,6 graphrag.index.graph.extractors.graph.graph_extractor ERROR error extracting graph