(Prototype) Multi-Agent AI UI Designer

[synapsecode]

PR Description

Hi, I have implemented the very early version of what I claimed I will do in my Proposal.
This Prototype PR demonstrates the following critical features that I have integrated into the APIDash Application itself:

• Modular Multi-Agent Calling Internal System
• Semantic Analysis & Intermediate Representation of API Response Creation
• SDUI Code Generation & Previewing Functionality
• Component Modification
• Component Code Export
• Settings UI Changes (to accept LLM Provider & API Keys)

Will try to prototype Tool generation too in the coming days.

This is an early version. The working Providers as of now are ollama and gemini. Ollama works but I need to add extra validators to prevent faulty generations (out of scope for this prototype). Gemini is working quite well.

I am excited to see improve and make this thing production ready in my GSoC period!

Related Issues

Prototype for Idea in: #617

Checklist

• I have gone through the contributing guide
• I have updated my branch and synced it with project main branch before making this PR
• I am using the latest Flutter stable branch (run flutter upgrade and verify)
• I have run the tests (flutter test) and all tests are passing

Added/updated tests?

We encourage you to add relevant test cases.

• Yes
• No, and this is why: This is just a simple prototype

OS on which you have developed and tested the feature?

• Windows
• macOS
• Linux

[synapsecode]

Full Video Demo can be seen here.
The Code is Executable too!

@animator @ashitaprasad

AI_UI_DESIGNER_PROTO.mp4

[DenserMeerkat]

Hey @synapsecode,
The initial implementation looks fine.

Since the UI Generation only considers the Response Body it would make more sense to place the Generate UI button within the "Response Body" tab alongside Copy and Download buttons.

I was able to successfully generate the UI output 2/5 times and I believe that can be improved by tuning the prompts?
I mostly ran into 503 and the below error on using Gemini as the provider.

Receiver: null
Tried calling: []("STAC")
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      _DialogContentsState.generateSDUICode (package:apidash/widgets/ai_ui_desginer_widgets.dart:77:20)
<asynchronous suspension>
#2      _DialogContentsState.build.<anonymous closure> (package:apidash/widgets/ai_ui_desginer_widgets.dart:108:26)
<asynchronous suspension>```

[synapsecode]

Hi Ragul, apologies for the delayed response.

1. Yes, good point. I will move the Generate UI button to the Response Body Tab

2. Certainly, the error rate can be improved a lot by prompting. Currently the system prompt for the Requirement -> SDUI and SDUI -> Flutter Code is basic. Doesnt include many examples and instructions. With a fairly basic prompt itself it is able to generate mostly well.
   Additionally, there is little to no validation logic, I have provisions in my code itself to write agent-specific validation logic. If validation fails, auto-retry is possible which can eliminate failures.

so with some good prompt engineering + adding relevant examples and correct validation logic, the success rate should definitely improve

Lastly, I looked up 503 issue on Gemini., that is something coming from server side gemini itself. "service unavailable" or "overloaded". Will add better error handling into the prototype so that the user is not confused

expect an updated POC by late EOD

[synapsecode]

Hi, @DenserMeerkat @animator

I have improved the POC and addressed the above points.

1. I have moved those buttons to the Response Body Tab however need more design guidance on how to fit them near the copy button as there is no space. So i have added it as an extra element on top

2. Have improved the Retry Logic to be more robust, System Prompt modifications and Validation logic implementations can be done during the GSoC period as it requires detailed analysis. With robust system prompts and logic, we can easily improve the accuracy of generated components

3. Have added Error handling for null checks and 503 errors. you should see an error snackbar now (greatly improved user experience)

Screen.Recording.2025-05-04.at.3.56.29.AM.mov

4. I have added API Tool Generation Feature (still in early stages) but as mentioned in my proposal I have implemented the function generator & tool generator bots
   Demo:

Screen.Recording.2025-05-04.at.3.52.26.AM.mp4

I have pushed the latest code to the same PR
Hopefully this addresses all your concerns, Lmk if you want to see anything else
Thanks!