Streaming with OpenAIChatCompletionClient raises an 'empty_chunk` exception when usage is requested, fixed using max_consecutive_empty_chunk_tolerance

[auto-d]

The model client documentation suggests this fix for missing tokens in an OpenAIChatCompletionClient streaming response:

set extra_create_args={"stream_options": {"include_usage": True}},

However the (final) message from the server with the requested usage information raises an exception due to an 'empty chunk' in the streaming processor implementation (openai/_openai_client.py).

Code to reproduce in autogen 0.41:

model_client = OpenAIChatCompletionClient(model="gpt-4o-mini", api_key = api_key)
model_client.create_stream(
                messages=["Tell me a story about pirates"],
                extra_create_args={"stream_options": {"include_usage": True}})

async for response in stream: 
    print(response) 
    # ValueError raised by logic in _openai_client.py's `create_stream`. 

Workaround for me was setting max_consecutive_empty_chunk_tolerance to 2, which per the comments was included to fix a problem with the Azure endpoint.

[ekzhu]

Could you submit a PR to update the API docs of the create_stream methods of both OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient on your error resolution? cc @MohMaz

[auto-d]

Could you submit a PR to update the API docs of the create_stream methods of both OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient on your error resolution? cc @MohMaz

@ekzhu both of those classes inherit their create_stream implementation from BaseOpenAIChatCompletionClient. Do you want the documentation updated there or in the respective class docs?

[ekzhu]

@auto-d , let's update the API doc of AzureOpenAIChatCompletionClient

[auto-d]

I'm not able to test the Azure behavior. PR'd changes to both classes as well as the user guide. Take what you will! :)

[ekzhu]

@auto-d thanks! Let's put it under the BaseOpenAIChatCompletionClient's create_stream method.