[Bug]: Too many agent responses lead to a laggy UI

[avi12]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

Every time I have a deep-ish conversation with the agent, the web page can barely respond
I suggest to modify the "chat" experience to lazy load the messages

OpenHands Installation

Docker command in README

OpenHands Version

main

Operating System

Windows

Logs, Errors, Screenshots, and Additional Context

No response

[mamoodi]

CC @rbren and @amanape

[amanape]

Could you explain what kind of laggy UI you are experiencing? Does it take a long time to load when switching between tabs? Is it problematic when scrolling?

[avi12]

Suppose I want to click on something, when I hover on the element it might take a few seconds for the cursor to turn into a pointer, and when I click it it will take a few seconds to register the click and then update the DOM

[amirshawn]

This is happening to me as well. I might be wrong but it seems like even if I clear out the long conversation and start a new one. Reload the page in a new browser window. It seems to start lagging on the second conversation a lot quicker than the first conversation.

[mamoodi]

Weird. I'll see if someone can take a look.

[avi12]

This is happening to me as well. I might be wrong but it seems like even if I clear out the long conversation and start a new one. Reload the page in a new browser window. It seems to start lagging on the second conversation a lot quicker than the first conversation.

Can confirm

[Jakan-Kink]

I'm not sure if the posters before me are using Claude 3.5, but for me after a little bit of time it takes more than a minute per response because of

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

23:04:45 - openhands:ERROR: retry_mixin.py:47 - litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}. Attempt #1 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

23:05:01 - openhands:ERROR: retry_mixin.py:47 - litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}. Attempt #2 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

23:05:16 - openhands:ERROR: retry_mixin.py:47 - litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}. Attempt #3 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

23:05:31 - openhands:ERROR: retry_mixin.py:47 - litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}. Attempt #4 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

23:05:48 - openhands:ERROR: retry_mixin.py:47 - litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}. Attempt #5 | You can customize retry values in the configuration.
23:06:26 - ACTION

It seems like Clause just keeps adding to the message all of the content it has already processed, and get to the point that just a simple prompt takes many, many minutes per step.

After looking at the other bugs, this might be a dupe of #5715

[avi12]

@Jakan-Kink Yep, OH easily runs into this issue
The only way to bypass it as far as I'm aware is by contacting Anthropic's sales department, which you can only do if you have a company email (i.e not of Gmail and possibly not Outlook, Yahoo, etc)
I just gave up and switched to GPT-4o and I haven't ran into this issue since

[amirshawn]

@Jakan-Kink you need to get to a higher tier through anthropic and the only way is to spend more. I think I had to buy $500 in tokens to get up to tier 4 and now I get 400,000 tokens/minute. I still hit that limit every 30 seconds or so and then it bogs down until the next minute. A nice feature to add to Open Hands would be a setting for how many input tokens you can use in a minute so instead of sending messages that won't work, Open Hands would throttle the conversation at that limit. Until Anthropic raises the limit to about 1,000,000 tokens per minute I don't think it will be able to catch up to Open Hands. I guess what I'm saying is this isn't a bug in Open Hands. The other post you linked to I shared a strategy that I think would also help us not reach that limit as quickly. The issue that's occurring in this bug report is about the length of the conversation causing too much memory to be used by the browser to where eventually the page is unresponsive.