Skip to content

Commit

Permalink
Fix a small bug in model value evaluation (#142)
Browse files Browse the repository at this point in the history
* Value evaluation feedback not correct. We should give different feedback when target tensor or gt tensor is not available.

* fix CI
  • Loading branch information
peteryang1 authored Aug 1, 2024
1 parent 6b5ad06 commit c14eafb
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions rdagent/components/coder/model_coder/CoSTEER/evaluators.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,10 @@ def value_evaluator(
prediction: torch.Tensor,
target: torch.Tensor,
) -> Tuple[torch.Tensor, bool]:
if target is None or prediction is None:
return "No output generated from the model. No value evaluation conducted.", False
if prediction is None:
return "No output generated from the model. Skip value evaluation", False
elif target is None:
return "No ground truth output provided. Value evaluation not impractical", False
else:
# Calculate the mean absolute difference
diff = torch.mean(torch.abs(target - prediction)).item()
Expand Down

0 comments on commit c14eafb

Please sign in to comment.