Clarified instructions for restaurants database

[rishsriv]

Previously, we had instructions like this:

Match city_name and food_type case-insensitively. Match with ILIKE and percent sign for substring matching for all other string matches.

This was confusing because both these lines were saying the same thing. I have changed this to Match all strings case-insensitively for all database types.

[wendy-aw]

Ok noted! Quick question, do you think we should keep the percent wildcard sign instruction since that hasn't got anything to do with case insensitivity?

[rishsriv]

Good point will fix post lunch!

[wongjingping]

Thanks for flagging this out. iirc case-insensitive match is slightly different from LIKE and percent sign match (or aka wildcard match). The former was generally used to refer to LOWER(column)='value', while the latter was used to refer to column LIKE '%value%'. Hence, "Match city_name and food_type case-insensitively. Match with LIKE and percent sign for substring matching for all other string matches." would that we match city_name and food_type with the former syntax (LOWER), while other columns like restaurant.name or location.street_name would use the latter syntax (LIKE).
That said, all of our questions would pass given existing data if we used the latter syntax (since city_name and food_type do not have additional prefixes/suffixes to the matched value).

I think we have 2 options on how to modify:

1. We can just specify the instructions for all of them to just use wildcards. This is the simplest, but doesn't test the model's ability to avoid using ILIKE and stick with LOWER.
2. We can keep the existing instructions, but should tweak the data such that we have data with extensions like food_type='neo-italian' that shouldn't be returned, so as to ensure that the model uses the former syntax (LOWER).

Lmk your thoughts!

[rishsriv]

Thanks for the clarifications!

I'm good with matching all strings case-insensitively for now. We should specify more granular and challenging instructions in the future (sp because our current benchmark is getting a bit saturated now with scores in/approaching 90+%). But that's a task for the future and not the best use of our time right now, methinks!

Let me know. I have modified the instructions to Match all strings case-insensitively using wildcard operators for clarity

[wendy-aw]

Okie gotcha!