[Bug]: OpenHands resolver fails to identify issues from new comments

[regismesquita]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

I am experiencing issues with the resolver when using it to address a comment in the PR. It seems to disregard my comment and instead follows the original issue. Interestingly, it eventually acknowledges that it did not address the concern raised in the comment.

This is a test issue where I requested a change in a Prometheus metric from 0.1 to 0.100. The project is in Go, and the OpenHands resolver modified the Go version.

OpenHands Installation

Other

OpenHands Version

openhands-resolver@main

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

The feedback may be addressed to specific code files. In this case the file locations will be provided.
Please update the code based on the feedback for the repository in /workspace.
An environment has been set up for you to start working. You may assume all necessary tools are installed.
# Issues addressed 
[
    "We are currently exporting to prometheus metrics from 0..1 , we want them to be 0..100"
]
# Review comments
None
# Review threads
None
# Review thread files
None
[...]
 12:09:34 - openhands:INFO: resolve_issue.py:210 - **IPythonRunCellAction**
THOUGHT: I'll help fix the issue where the Prometheus metrics need to be scaled from 0..1 to 0..100. Let me first explore the repository to find relevant files that handle Prometheus metrics.
CODE:
print(file_editor(**{'command': 'view', 'path': '/workspace'}))

And by the end:

openhands:ERROR: resolve_issue.py:262 - Failed to parse success_explanation as JSON: The AI agent's response did not address the feedback given in the PR thread comment, which was specifically about removing the .bak file and not changing the Go version. Instead, the agent's response focused on analyzing the metrics ranges, which was from the original issue description.

[neubig]

Here is a thread on the OpenHeads github where we were able to reproduce:

• thread: build(deps-dev): bump the eslint group in /frontend with 6 updates #5098
• github action: https://github.com/All-Hands-AI/OpenHands/actions/runs/11901277063/job/33163889631

[neubig]

Probably fixed by #5115