🚀 AutoGPT Roadmap - Web interaction 🦾🖥️

[Pwuts]

Objective 🎯

AutoGPT should be able to interact with web services to achieve any in-scope, allowed-by-design outcome that a human could achieve through interacting with the website.

SWOT 🧭

Strengths 💪

• Existing Selenium+Chrome integration with ad blocker and cookie wall squasher
• Code execution environment available

Weaknesses 🚧

• Selenium+Chrome devours RAM
• No visual browsing yet

Opportunities 🚀

• API discovery
• Parallel multi-path exploration

Threats ⚠️

• Every LLM operation costs $$; LLM-intensive workflow could be expensive

Related 👪

• 🚀 AutoGPT Roadmap - Ability Acquisition 💡 #6965

Actionables

• Goal-oriented browser interaction sub-agent
  • Let the “main agent” specify intent, website, and context (e.g. credentials) if applicable
  • Pass (intent, website, context) to an agent that can directly and effectively interact with the browser
  • Browsing agent reports back to main agent every X steps and/or when it finishes
    • Main agent must approve to continue every X steps and can provide feedback
• API finder and client generator module
  • If a website has an API, we can find it in an automated way and possibly use it instead of the web interface