Gemini does not respect the order of the properties in a schema

[seungduk-yanolja]

Description of the bug:

Let's imagine a situation where you want an answer and a rationale from the LLM. You want it to be in a JSON format like

{
  "choice": 4,
  "rationale": "The answer is 4 because..."
}

If the LLM completes the choice property first, then the next rationale can be made up to justify the choice.
However, if the rational comes first like

{
  "rationale": "The answer is 4 because...",
  "choice": 4
}

Then, the choice will be determined by rationale. It is similar to CoT.

It is important that the LLM follows the given order of the properties in an object. However, I found that Gemini SDK ignores it.

    properties: MutableMapping[str, "Schema"] = proto.MapField(
        proto.STRING,
        proto.MESSAGE,
        number=3,
        message="Schema",
    )

I think this one better to be a repeated, not a map field, to keep the order of the properties.

Actual vs expected behavior:

Gemini SDK ignores the order of the properties in an object.

Any other information you'd like to share?

No response

[seungduk-yanolja]

Any updates? also there is another issue:
google/generative-ai-go#211

[httplups]

I am having the same issue that Gemini ignores the order; it tends to hallucinate

[httplups]

Hi. I was able to make the model follow the order by changing the field names to 'step_one' and 'step_two'. It is not ideal, but its a workaround to have an structured output with reasoning.