You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The definition hints are unquestionably an improvement over the dictionary definitions for use in hint panels, which is great! That being said, they could definitely be improved. The part of the instructions concerning how to generate hints is as follows:
Each hint should be one or two sentences and should not contain any form of the word you're providing a hint for, or any word or phrase that the submitted word can be an abbreviation for. For example, the hint for "allay" should not contain "allay," "allaying," "allays," etc. The hints should focus on the meaning of the word, although hints about the pronunciation are also acceptable. These hints are designed to replicate the user-provided hints that users provide on the New York Times Spelling Bee forums for words in that day's Spelling Bee puzzle.
The issues with the current hint generation are:
They are still more "definition" than "hint." While they give much less away now, it would be great if they were more tailored to hinting at the word rather than just giving a definition. For example, in the spelling bee forums, someone provided a hint for the words "eave" and "eaved" for one puzzle with Night before "roof overhang day"? This combines clues about both the pronunciation (via a homonym) as well as the definition into one succinct, effective hint. The AI can't generate anything that "clever" yet, but it might be able to with proper instructions or fine-tuning. Essentially, the goal is crossword-style hints that combine clues about the definition, pronunciation, and potentially context clues like other words in the puzzle to generate more interesting and varied hints.
Even when following the guidelines above, many hints still give away too much information. For example, the hint for "vale" is "A valley." Technically, "valley" and "vale" are different words, so providing a very similarly spelled and pronounced synonym isn't breaking the rules about not providing any form of the word, but... it would be better if the hint was something else. This might be fixable by tweaking the instructions.
The AI sometimes just straight-up ignores the instructions, such as not to include any form of the word in the hint. For example, the hint for "uvulae" reads "Plural of 'uvula'; fleshy extensions at the back of the soft palate." The hint for "valuate" is "To determine or estimate the value of something, especially in a financial context." This might be fixable by slightly tweaking the instructions to "...should not contain any form of the word you're providing a hint for or that word's root word, ...", but it's also possible that some amount of this instruction non-adherence is just inherent to AI models, in which case the issue could likely be mitigated somewhat by switching to a newer model (e.g., GPT 4 instead of the current model of GPT 3.5).
There will likely need to be iterative improvement on the hint generation logic before the hints are consistently generated correctly.
At a high level, this iterative improvement could include any of the following elements:
UI functionality for marking words that have unacceptable hints and regenerating them manually (likely in batches)
Using GPT 4 going forward and/or regenerating all existing hints using GPT 4 instead of GPT 3.5
Refining the instructions for generating hints
Fine-tuning an AI model for hint generation by giving it examples of good and bad hints
The text was updated successfully, but these errors were encountered:
As described in PR #121:
The definition hints are unquestionably an improvement over the dictionary definitions for use in hint panels, which is great! That being said, they could definitely be improved. The part of the instructions concerning how to generate hints is as follows:
The issues with the current hint generation are:
Night before "roof overhang day"?
This combines clues about both the pronunciation (via a homonym) as well as the definition into one succinct, effective hint. The AI can't generate anything that "clever" yet, but it might be able to with proper instructions or fine-tuning. Essentially, the goal is crossword-style hints that combine clues about the definition, pronunciation, and potentially context clues like other words in the puzzle to generate more interesting and varied hints.There will likely need to be iterative improvement on the hint generation logic before the hints are consistently generated correctly.
At a high level, this iterative improvement could include any of the following elements:
The text was updated successfully, but these errors were encountered: