Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent Developer Role Requirement for o1 and o1-mini Models #330

Open
coolnitin157 opened this issue Jan 29, 2025 · 1 comment
Open
Labels
bug Something isn't working

Comments

@coolnitin157
Copy link

coolnitin157 commented Jan 29, 2025

I encountered an issue where the o1-mini model throws an error stating that it "does not support system role," while the same code works perfectly for the o1 model without requiring the Developer role. As per the official OpenAI documentation, reasoning models should require the Developer role for system messages. However:

Expected Behavior:

Both o1 and o1-mini should either consistently accept system messages without requiring a Developer role or reject them in accordance with the documented behavior.
Actual Behavior:

The o1-mini model fails with an error regarding the unsupported system role.
The o1 model works successfully without needing a Developer role, contrary to expectations.

Steps to reproduce

Steps to Reproduce:

Use the o1-mini model with system messages in the payload.
Observe the error: "Model does not support system role."
Switch to the o1 model and note that it works without errors, despite no Developer role assignment.

Code snippets

var messages = new List<ChatMessage>
{
    new SystemChatMessage("Some system message here"),
    new UserChatMessage("asked question with dependency on system message")
};

try
{
    var defaultOpenAIKey = Environment.GetEnvironmentVariable("openAIKey", EnvironmentVariableTarget.Process);
    var openAiClient = new ChatClient("o1-mini", defaultOpenAIKey);
    var result = await openAiClient.CompleteChatAsync(messages, options, cancellationToken);
    return result.Content[0].Text;
}
catch (Exception ex)
{
    return new BadRequestObjectResult(ex.Message);
}

OS

windows

.NET version

8

Library version

2.1.0

@coolnitin157 coolnitin157 added the bug Something isn't working label Jan 29, 2025
@coolnitin157
Copy link
Author

o3 mini works fine with above code, just tested it today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant