Set max tokens by prompt

[prasmussen15]

• The default max tokens is reduced to 2048 for all prompts
• Edge extraction, the prompt with the largest potential output, has had its max output tokens overridden to 16352

---

Important

Set default max tokens to 1024 for LLM prompts, with 16384 for edge extraction, and updated Dockerfile and version.

• Behavior:
  • Default max_tokens reduced to 1024 in config.py.
  • Edge extraction in edge_operations.py has max_tokens set to 16384.
• Functions:
  • Added max_tokens parameter to _generate_response() and generate_response() in client.py, anthropic_client.py, groq_client.py, openai_client.py, and openai_generic_client.py.
• Misc:
  • Updated Dockerfile to use --only main --no-root for Poetry installation.
  • Bumped version to 0.5.2 in pyproject.toml.

^{This description was created by for 1083bbe. It will automatically update as commits are pushed.}

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 1d58efa in 1 minute and 20 seconds

More details

• Looked at 229 lines of code in 7 files
• Skipped 0 files when reviewing.
• Skipped posting 5 drafted comments based on config settings.

1. graphiti_core/llm_client/config.py:17

• Draft comment:
  The DEFAULT_MAX_TOKENS value is set to 1024 here, but other client files like anthropic_client.py, groq_client.py, and openai_client.py have different default values for DEFAULT_MAX_TOKENS. Consider consolidating these values to avoid inconsistencies.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The change in DEFAULT_MAX_TOKENS from 16384 to 1024 in config.py is not reflected in the anthropic_client.py, groq_client.py, and openai_client.py files where DEFAULT_MAX_TOKENS is defined separately. This could lead to inconsistencies if the intention was to have a uniform default across all clients.

2. graphiti_core/llm_client/anthropic_client.py:65

• Draft comment:
  Ensure max_tokens is not None before using it in max_tokens=max_tokens or self.max_tokens. This logic could lead to unexpected behavior if max_tokens is explicitly set to None.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The max_tokens parameter is being passed with a default value of DEFAULT_MAX_TOKENS, but the logic max_tokens=max_tokens or self.max_tokens could lead to unexpected behavior if max_tokens is explicitly set to None. This should be handled to ensure the correct default is used.

3. graphiti_core/llm_client/groq_client.py:64

• Draft comment:
  Ensure max_tokens is not None before using it in max_tokens=max_tokens or self.max_tokens. This logic could lead to unexpected behavior if max_tokens is explicitly set to None.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The max_tokens parameter is being passed with a default value of DEFAULT_MAX_TOKENS, but the logic max_tokens=max_tokens or self.max_tokens could lead to unexpected behavior if max_tokens is explicitly set to None. This should be handled to ensure the correct default is used.

4. graphiti_core/llm_client/openai_client.py:108

• Draft comment:
  Ensure max_tokens is not None before using it in max_tokens=max_tokens or self.max_tokens. This logic could lead to unexpected behavior if max_tokens is explicitly set to None.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The max_tokens parameter is being passed with a default value of DEFAULT_MAX_TOKENS, but the logic max_tokens=max_tokens or self.max_tokens could lead to unexpected behavior if max_tokens is explicitly set to None. This should be handled to ensure the correct default is used.

5. examples/podcast/podcast_runner.py:59

• Draft comment:
  Remove commented-out code if it's not needed to keep the codebase clean and maintainable.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The commented-out code should be removed if not needed, as it violates the DRY principle and clutters the codebase.

Workflow ID: wflow_aupVXIxl0a9Hpcb1

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 329e103 in 22 seconds

More details

• Looked at 75 lines of code in 3 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. graphiti_core/llm_client/openai_generic_client.py:114

• Draft comment:
  The max_tokens parameter is not being used in _generate_response. Replace self.max_tokens with max_tokens to use the dynamic value passed to the function.
• Reason this comment was not posted:
  Comment was not on a valid diff hunk.

2. examples/podcast/podcast_runner.py:56

• Draft comment:
  The main function is doing multiple tasks: setting up logging, clearing data, building indices, and processing messages. Consider refactoring to separate these concerns into distinct functions.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The main function in podcast_runner.py is performing multiple tasks, which violates the Single Responsibility Principle. It should be refactored to separate concerns.

Workflow ID: wflow_o5eFqZUoWsQ6nPR2

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 70e0f0a in 13 seconds

More details

• Looked at 19 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. graphiti_core/helpers.py:76

• Draft comment:
  Consider specifying the return type for normalize_l2 for clarity and consistency with the rest of the codebase.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The function signature change in normalize_l2 is not consistent with the rest of the codebase. The return type should be specified for clarity and consistency.

2. graphiti_core/helpers.py:76

• Draft comment:

 def normalize_l2(embedding: list[float]) -> list[float]:

• Reason this comment was not posted:
  Confidence changes required: 80%
  The function normalize_l2 has a return type hint that is missing. It should specify the return type for clarity and consistency with Python typing standards.

Workflow ID: wflow_xTYD6CEaC6fyfk3U

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 1083bbe in 13 seconds

More details

• Looked at 13 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. Dockerfile:26

• Draft comment:
  Ensure that using --only main --no-root instead of --no-dev does not omit any necessary dependencies for the server to run correctly. This change alters the scope of installed dependencies.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The change in the Dockerfile from --no-dev to --only main --no-root is significant. It changes the scope of dependencies being installed, which could affect the runtime environment if any necessary dependencies are omitted. This should be verified against the project's requirements.

Workflow ID: wflow_aoBnuEF5LUNt9e6M

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.