You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Issue Summary generates 2-3 sections currently: what's wrong, possible cause, and what's in the trace (and potentially more in the future, like session replays). We already ask the model to only generate the sections it has something for, so you'll see the trace section missing sometimes.
Still, the model, if it doesn't have the right information, often just makes a bold and incorrect guess, especially for the other sections. We think an indicator of the confidence of each generated section would make sense here, so that we can hide them from the user if we're not confident in the quality.
(UX of hiding vs. just showing the score still needs to be agreed upon)
At the end of the day, what we're trying to solve for is not providing developers with incorrect or misleading information. If you see an approach better than a confidence score for solving this problem, that would be great!
The text was updated successfully, but these errors were encountered:
Issue Summary generates 2-3 sections currently: what's wrong, possible cause, and what's in the trace (and potentially more in the future, like session replays). We already ask the model to only generate the sections it has something for, so you'll see the trace section missing sometimes.
Still, the model, if it doesn't have the right information, often just makes a bold and incorrect guess, especially for the other sections. We think an indicator of the confidence of each generated section would make sense here, so that we can hide them from the user if we're not confident in the quality.
(UX of hiding vs. just showing the score still needs to be agreed upon)
At the end of the day, what we're trying to solve for is not providing developers with incorrect or misleading information. If you see an approach better than a confidence score for solving this problem, that would be great!
The text was updated successfully, but these errors were encountered: