Empty context causes 4o models to return "N/A"

[excubo-jg]

if dspy.Prediction is called with context = [] this causes in chat_adapter.py

def format_fields(fields):
    output = []
    for k, v in fields.items():
        v = _format_field_value(v)
        output.append(f"[[ ## {k} ## ]]\n{v}")

and
def format_list(items): if len(items) == 0: return "N/A"

that content in format_turn()
content.append(format_fields({k: values.get(k, "Not supplied for this particular example.") for k in field_names}))
is set to
['[[ ## context ## ]]\nN/A\n\n[[ ## question ##]]\nMy prompt']

This context makes OpenAI 4o (but not 4 or older versions) reply "N/A" to all prompts. I think the line returning "N/A" should be changed to return "" - could do a pr if this is confirmed.

[okhat]

Thanks for the note, @excubo-jg ! Can you share an example of this happening?

[okhat]

For example, I just tried this with GPT-4o:

dspy.Predict('context, question -> answer')(context=[], question='What is the capital of France?')

The answer I get is "The capital of France is Paris."

[excubo-jg]

This is getting a bit weird... Using your question I get "Paris" as answer.
Some other examples:
"What are the first steps to connect to SAP?" -> "Install SAP GUI"
"What are the first steps to connect to SAP ARIBA?" -> "N/A"
I was using non-sensical prompts like "sdvsvders?" during testing of my other code which return "N/A" with 4o. With 4 I get "Could you please clarify your question?"

[okhat]

I'm not seeing any issues in behavior. I'm using DSPy 2.5.6 (latest). I tried both 4o and 4o-mini. Both look good.

Example 1:

dspy.Predict('context, question -> answer')(context=[], question='sdvsvders?')
dspy.inspect_history()

Produces:

Prediction(
    answer="I'm sorry, but it seems like your question is incomplete or unclear. Could you please provide more context or clarify your question?"
)

Example 2:

dspy.Predict('context, question -> answer')(context=[], question='What are the first steps to connect to SAP ARIBA?')

Produces:

Prediction(
    answer="To connect to SAP Ariba, the first steps typically involve:\n\n1. **Account Setup**: Ensure you have an SAP Ariba account. If not, you will need to register on the SAP Ariba Network.\n\n2. **Access Credentials**: Obtain the necessary credentials, such as a username and password, or API keys if you are connecting programmatically.\n\n3. **Network Configuration**: Configure your network settings to allow communication with SAP Ariba. This may involve setting up VPNs or adjusting firewall settings.\n\n4. **Integration Setup**: If integrating with other systems, configure the necessary integration points. This could involve setting up middleware or using SAP Ariba's APIs.\n\n5. **Testing**: Conduct initial tests to ensure connectivity and data exchange are functioning as expected.\n\n6. **Training and Documentation**: Familiarize yourself with SAP Ariba's interface and functionalities through available training resources and documentation.\n\n7. **Go Live**: Once testing is successful, proceed to go live with your SAP Ariba connection.\n\nThese steps may vary slightly depending on your specific use case and the SAP Ariba solutions you are implementing."
)

[excubo-jg]

Not sure what is going on - there must be side effects in my processing. Many thanks for your kind support.