[Feature]: Aider-inspired RepoMap

[neubig]

What problem or use case are you trying to solve?

Aider has a functionality to create a RepoMap, which is a concise description of the repo in text format, with the most relevant/important parts highlighted.

Describe the UX of the solution you'd like

It would be nice to have a RepoMap class within OpenDevin that can be used by any agent to pull in a description of the repo.

Do you have thoughts on the technical implementation?

• copy the aider repomap.py into OpenDevin. This could probably be put in either a new indexing folder here, or in the memory folder.
• make it possible to import the repomap into one of our competitive agents, such as CodeAct. This functionality should be optional, like the github message (details).
• run experiments on SWE-Bench to see if this functionality improves accuracy

Describe alternatives you've considered

We could also implement this from scratch, or create improved code search functionality.

Additional context

• Parent issue Create Aider Agent #120

[neubig]

@ryanhoangt will probably take a look at this

[tobitege]

Reference: #2248 Add Aider-inspired RepoMap

[0xdevalias]

Describe alternatives you've considered

We could also implement this from scratch, or create improved code search functionality.

See also:

• Explore using stack graphs for better code search / navigation / context / repo map / etc #742

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[0xdevalias]

IMO this shouldn't be closed as stale

[FellowTraveler]

repomap and graphrag are critical features IMO for coding agents.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[TomLucidor]

Is this the reason why when the GitHub repo is too large, OpenHands will hang? That there are no systematic way of understanding/ignoring files and folders (including extensions)?

[neubig]

Hey @TomLucidor, sorry about the trouble!

If you have a large repo OpenHands should not hang, although it might become confused. The hanging behavior is likely due to something else. We'd be happy to help diagnose this, but in order to do so we'll need to be able to reproduce the problem. The easiest way for us to do this is if you can press the "feedback" button when you're encountering this behavior, or share logs. If you can do that it'd be great if you can open a new issue. I'd also be happy to discuss on slack.

[xingyaoww]

Hey @ryanhoangt can you self-assign this one?

[ryanhoangt]

Sure!

[BradKML]

As a little heads up, has anyone checked these repos out? (something along the lines of "codebase prompt")

• https://github.com/cyclotruc/gitingest
• https://github.com/mufeedvh/code2prompt
• https://github.com/raphaelmansuy/code2prompt
• https://github.com/kamilstanuch/codebase-digest

[enyst]

I found this kind of tool helpful, but not for feeding it as such to openhands, more for local work in or alongside vscode/cursor. It works for large codebases, to dump into Gemini, and go.

It's not great (Gemini is not great, at least pre-2.0), but I get something out of it. For now, I doubt it has the precision to be useful for the agent, considering the tradeoffs here. Have you tried to give it like that, in a message?

[BradKML]

@enyst what do you have in mind compared to the established tools made specifically for chatbots? Cus I am tempted but not even sure whether to put/generate it in a text file for the repo, OR to throw it directly in the dialogue box. Asking second opinions

[ryanpeach]

Just wanted to point out, you can import aider.repomap and get this feature for free.

[ryanpeach]

@rbren where is the PR that completed this?

[raymyers]

Just noting here that @xingyaoww discussed the reason for closing this issue in today's community meetup (Jan 31, 2025), so until more info is added here, it should be in the recording about 15 min in.

[xingyaoww]

I believe @ryanhoangt can explain more about this, but in summary, we've been trying something like this: All-Hands-AI/openhands-aci#41 for integrating Aider-inspired Repo map -- and it seems we didn't get it working on claude after a few attempts (e.g., performance on SWE-Bench didn't improve when adding those file skeleton) -- hoang feel free to correct me if I'm wrong, and it will be nice if you can share some numbers too!

I found this kind of tool helpful, but not for feeding it as such to openhands, more for local work in or alongside vscode/cursor. It works for large codebases, to dump into Gemini, and go.

As Engel said, there's a significant trade-off here:

• For single-turn systems where the LLM typically doesn't have the autonomy to look around and execute actions (e.g., I need to paste context to ChatGPT/Claude for it to generate another block of code), such type of RepoMap is essential -- an example of this would be Aider that primarily focuses on "file-editing" action, as opposed to executing general bash command like "grep -n".
• However, for a flexible multi-turn agentic system like OpenHands (CodeAct) -- sometimes adding more of this info actually did more harm than good. Basically, if you don't tell the agent anything, it would think, "I need to poke around and figure out right context,"; but if you tell it something like RepoMap (but it could be incomplete!) -- the agent would trust your RepoMap and think that's everything they need to know to complete the task. Here's another bitter lesson that happened recently (fix: revert #5506 for SWE-Bench performance regression #6491): we think providing less context of the file tree structure to the agent would help, but it actually hurt things a lot - so definitely, "what humans think is useful might not hold true for autonomous multi-turn agents" - and that's why we depends on evals for different design decisions!

I would think things like RepoMap were initially created to build context for a "single-turn LLM-based system."
However, they may not be useful for multi-turn systems like OpenHands, because the agent has the autonomy to look for stuff they need -- and a lot of times, it did better than "our human-curated repo map based on our intuition of what the model needs" -- another vivid reminder of the bitter lesson 😢 .

Here are some baseline code localization performance numbers on a paper I've been working on with some folks (haven't released yet! i can share the preprint here when we get it online):

You can see even if we don't have any system like "RepoMap" to manually construct context within a large repo - whereas other systems like "agentless" and "moatless" all have RepoMap-style systems - they didn't really outperform OpenHands on everything (if not worse!).

Based on these existing results, I've personally feel it is probably less productive to keep pushing towards this direction compared to other things after spending a lot of effort here -- but I could be totally wrong, and would be open to re-open this PR if people from the community is interested in taking over and actually making it work ❤

[xingyaoww]

cc @jimwhite -- thanks for the great discussion in today's community meeting! 👆 hopefully this response above would better answer your question :)

[aymannadeem]

Hey @xingyaoww! You mentioned that agents often do better when left to explore versus being given a repo map. Could you share some specific examples where the agent's autonomous exploration led to better solutions than when provided with structured context?

[BradKML]

I am thinking something like eza can make dynamic folder maps https://github.com/eza-community/eza
Right now when I am testing OH often times the agent would forget the pwd or context of where the terminal is, after cding in or out, which is annoying and warrant some kind of tool that shows both ls and hints at pwd.

[enyst]

often times the agent would forget the pwd or context of where the terminal is, after cding in or out, which is annoying and warrant some kind of tool that shows both ls and hints at pwd.

The LLM is already supposed to get in the prompt, after each command, the current directory:

OpenHands/openhands/events/observation/commands.py

Line 159 in fe8b927

ret += f'\n[Current working directory: {self.metadata.working_dir}]'

If that is not happening, maybe you could post a new issue, with some log with it missing or wrong? But I think it's also possible that we send it and sometimes the LLM ignores it anyway (depending on LLM).

[BradKML]

It might be an LLM issue then @enyst since I am heavily leaning on DeepSeek v3 atm, but it can't invoke the observation whenever it moves around working directories it seems?

[enyst]

It's automatic. The LLM asks for a command to execute ('cd ...'), the environment performs it and responds with an observation which includes cwd as above.

[ryanpeach]

Very interesting writeup @xingyaoww