[Bug]When instance_id contains uppercase letters, running eval_infer results in an error.

[lycfight]

Some instance_ids in SWE-Gym contain uppercase letters, and the result files are consistent with them. Converting them to lowercase here would cause issues when reading the files. I understand that the reason for converting to lowercase is only because uppercase letters in the image name would cause an error, so the repository name does not need to be converted.

I propose a modification that allows correctly evaluating instance_ids containing uppercase letters:

from
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L307
to
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L310
remove:

  # if there's capital letters in the repo name, convert to lowercase
  if instance_id != instance_id.lower():
      print(f"Instance ID {instance_id} contains capital letters. Converting to lowercase.")
      instance_id = instance_id

https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L310
remove .lower():
repo = instance["repo"]

https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L257
remove .lower():
MAP_REPO_VERSION_TO_SPECS[instance["repo"]][instance["version"]]["test_cmd"]

from
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/constants.py#L2023
to
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/constants.py#L2027C1-L2027C60
remove:

# All keys should be in lower case
LOWER_MAP_REPO_VERSION_TO_SPECS = {
    k.lower(): v for k, v in MAP_REPO_VERSION_TO_SPECS.items()
}
MAP_REPO_VERSION_TO_SPECS = LOWER_MAP_REPO_VERSION_TO_SPECS

from
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/log_parsers.py#L373
to

SWE-Bench-Fork/swebench/harness/log_parsers.py

Line 375 in 242429c

MAP_REPO_TO_PARSER = LOWER_MAP_REPO_TO_PARSER

remove:

# All keys should be in lower case
LOWER_MAP_REPO_TO_PARSER = {k.lower(): v for k, v in MAP_REPO_TO_PARSER.items()}
MAP_REPO_TO_PARSER = LOWER_MAP_REPO_TO_PARSER

https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L82
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L86
https://github.com/SWE-Gym/SWE-Bench-Fork/blob/242429c188fcfd06aad13fce9a54d450470bf0ac/swebench/harness/test_spec.py#L87
add .lower():
self.instance_id..lower()