Fix file descriptor leak

[tofarr]

End-user friendly description of the problem this fixes or functionality that this introduces

This is a revert of #6887 (Which is itself a revert) with a fix for the log completions.

• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

---

Give a summary of what the PR does, explaining any non-trivial design decisions

• The log_completions functionality was not covered by any tests - so I added a test to cover this.
• I have excluded the client from the logged kwargs
• Taking the kwargs passed to litellm and converting them to json seems brittle (given that there is no guarantee that these arguments are jsonifiable), but the unit test should cover this in future.

---

Link of any specific issues this addresses

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:d802de9-nikolaik   --name openhands-app-d802de9   docker.all-hands.dev/all-hands-ai/openhands:d802de9

[enyst]

I think the first issue is fixed by this PR, I'm not sure about the other two. I'd like to test this with:

• Claude via litellm_proxy
• o3_mini directly from openai

It seemed to me that litellm does some duck typing undercover and expects this client kwarg to support other methods for those providers. 🤔

[tofarr]

I think the first issue is fixed by this PR, I'm not sure about the other two. I'd like to test this with:

• Claude via litellm_proxy
• o3_mini directly from openai

It seemed to me that litellm does some duck typing undercover and expects this client kwarg to support other methods for those providers. 🤔

I can confirm that this works with Claude via LiteLLM proxy. I'll test with o3_mini this afternoon

[enyst]

Weird, Claude via AH litellm_proxy, console:

15:27:13 - openhands:ERROR: loop.py:22 - RuntimeError: There was an unexpected error while running the agent. Please report this error to the developers. Your session ID is default. Error type: APIError
Traceback (most recent call last):
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 726, in completion
    raise e
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 643, in completion
    api_key=openai_client.api_key,
            ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'HTTPHandler' object has no attribute 'api_key'

which becomes

  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2202, in exception_type
    raise e
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 452, in exception_type
    raise APIError(
litellm.exceptions.APIError: litellm.APIError: APIError: Litellm_proxyException - 'HTTPHandler' object has no attribute 'api_key'

o3-mini via openai, console:

File "/Users/enyst/repos/odie/openhands/llm/llm.py", line 243, in wrapper
    resp: ModelResponse = self._completion_unwrapped(*args, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1190, in wrapper
    raise e
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1068, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/main.py", line 3085, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2202, in exception_type
    raise e
  File "/Users/enyst/repos/odie/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 452, in exception_type
    raise APIError(
litellm.exceptions.APIError: litellm.APIError: APIError: OpenAIException - 'HTTPHandler' object has no attribute 'api_key'
15:09:08 - openhands:INFO: agent_controller.py:466 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.ERROR

Maybe I'm doing something wrong, I'll try main. It just looks to me like litellm may expect some other client, also this comment on your litellm PR suggests there may be different clients for providers?

[enyst]

main, Claude via proxy, console run, looks good:

ACTION
[Agent Controller default] **MessageAction** (source=EventSource.AGENT)
CONTENT: What would you like me to help you with? Please let me know your task and I'll assist you accordingly. Also, if you encountered any errors previously, please let me know about them so we can address them properly.

[tofarr]

• o3_mini directly from openai

I see errors on this too - It looks like in some cases they expect the client object to be a HTTPHandler, and in others a different object type. That makes this... messy.

[tofarr]

@enyst - I have decided to come at this problem from a different direction - tracking the httpx client and closing it manually. It is an... unconventional approach, but I can see no other way aside from forking litellm, which I feel will add even more technical debt. (This fix will be transparent when the fix occurs, and can be removed at any later convenient time.)

I can confirm that this solution works when using:

• Claude directly
• os3 mini directly
• Claude via proxy

I can also confirm that it does not leak file descriptors.

[enyst]

I didn't test, but I think you're right this is the approach we need to take here. We've done some stuff like this before and hey, there will be some cases we'll do it again. If you think it's worth doing, let's go for it.