Skip to content

MiuLab/MultiAgent-Survey

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

37 Commits
 
 
 
 
 
 

Repository files navigation

Creativity in LLM-based Multi-Agent Systems: A Survey

arXiv Badge License Badge

Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts. While existing surveys provide comprehensive overviews of MAS infrastructures, they largely overlook the dimension of creativity, including how novel outputs are generated and evaluated, how creativity informs agent personas, and how creative workflows are coordinated.

This survey offers a structured framework and roadmap for advancing the development, evaluation, and standardization of creative MAS.

🔥 Update

🤝 Citation

@ARTICLE{10531702,
  author={Zhou, Xingcheng and Liu, Mingyu and Yurtsever, Ekim and Zagar, Bare Luka and Zimmer, Walter and Cao, Hu and Knoll, Alois C.},
  journal={IEEE Transactions on Intelligent Vehicles}, 
  title={Vision Language Models in Autonomous Driving: A Survey and Outlook}, 
  year={2024},
  pages={1-20},
  keywords={Autonomous vehicles;Task analysis;Planning;SData models;Surveys;Computational modeling;Visualization;Vision Language Model;Large Language Model;Autonomous Driving;Intelligent Vehicle;Conditional Data Generation;Decision Making;Language-guided Navigation;End-to-End Autonomous Driving},
  doi={10.1109/TIV.2024.3402136}}

📃 Introduction

MAS comprises multiple autonomous entities: software agents, robots, or human-AI hybrids. This structure enables emergent collaboration and richer exploration of open-ended creative spaces, significantly underlying the practice for computational creativity. Here, computational creativity refers to the creation of artifacts—such as ideas, behaviors, or solutions—that are both novel and valuable, demonstrating clear usefulness or appeal rather than being random.

This survey focuses on systems whose inputs and outputs span text and images. We aim to map the current landscape of techniques, datasets, evaluations, and challenges to foster and measure creativity in such multimodal and heterogeneous systems. By analyzing how different agents interact, we reveal how collaborative structures can unlock creative potentials that exceed what isolated LLMs or individuals can achieve.

🌟 MAS Workflow and Proactivity

Overall Proactivity across Process and Decision-Making

Method Year Process Decision-Making
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models arxiv(2024) low low
AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas CHIWork 2024 low low
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models arxiv(2024) low low
StoryDrawer: A Child–AI Collaborative Drawing System to Support Children's Creative Visual Storytelling CHI 2022 low low
CharacterMeet: Supporting Creative Writers' Entire Story Character Construction Processes Through Conversation with LLM-Powered Chatbot Avatars CHI 2024 low low
SPARKIT: A Mind Map-Based MAS for Idea Generation Support EMAS 2024 medium low
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANs The Journal of Supercomputing 2024 medium low
Towards Inclusive Co-creative Child-robot Interaction: Can Social Robots Support Neurodivergent Children's Creativity? HRI 2025 medium low
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping arxiv(2024) medium low
ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation 2024 medium low
Scideator: Human-LLM Scientific Idea Generation Grounded in Research-Paper Facet Recombination arxiv(2025) medium low
C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language Model CHI 2024 medium low
Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills arxiv(2025) medium low
Creative Wand: A System to Study Effects of Communications in Co-Creative Settings arxiv(2023) medium low
AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation CHI 2024 medium low
ChainBuddy: An AI-assisted Agent System for Generating LLM Pipelines CHI 2025 medium low
CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent GitHub CHI 2024 medium low
An Interactive Co-Pilot for Accelerated Research Ideation HCINLP 2024 medium low
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design IEEE Transactions on Visualization and Computer Graphics 2023 medium medium
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling CHI 2024 medium medium
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas arxiv(2024) high low
DesignGPT: Multi-Agent Collaboration in Design ISCID 2023 medium medium
Towards an AI co-scientist arxiv(2025) medium high
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation arxiv(2024) medium high
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worlds ICCC 2024 medium high
AgentReview: Exploring Peer Review Dynamics with LLM Agents GitHub EMNLP 2024 medium high
StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning FDG 2024 medium high
PersonaGym: Evaluating Persona Agents and LLMs GitHub arXiv(2025) high medium
AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaboration arxiv(2024) high medium
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis arxiv(2025) medium high
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming arxiv(2025) high high
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System GitHub arxiv(2025) high high
Generative Agents: Interactive Simulacra of Human Behavior GitHub UIST 2023 high high
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing EMNLP 2024 high high
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play GitHub arxiv(2024) high high
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate GitHub EMNLP 2024 high high
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration GitHub arxiv(2024) high high
Systematic Idea Refinement for Machine Learning Research Agents Tsinghua University Course:AML 2024 high high
Unlocking AI Creativity: A Multi-Agent Approach with CrewAI Journal of Trends in Computer Science and Smart Technology 2024 high high
On Generative Agents in Recommendation SIGIR 2024 high high
MARG: Multi-Agent Review Generation for Scientific Papers GitHub arxiv(2024) high high

🌟 MAS Techniques for Creativity

Divergent Exploration

Method Year Task Code Link
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking 2024 AUT and RAT
PersonaGym: A Reinforcement Learning Environment for Training Multi-Persona Agents 2024 Character Design
Creativity Support in the Age of Large Language Models 2024 Creative Writing
One Does Not Simply “Meme” Alone: Self-Dialogue for Creative Ideation 2024 Humor Co-Creation
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With LLMs 2024 Idea Generation
AI-Augmented Brainwriting: Exploring Idea Flow Between Groups and Large Language Models 2024 Idea Generation
AI and the Future of Collaborative Work: Group Ideation With an LLM in a Virtual Canvas 2024 Idea Generation
ContextCam: Bridging Context Awareness With Creative Human-AI Image Co-creation 2024 Image Generation
C2Ideas: Supporting Creative Interior Color Design Ideation With an LLM 2024 Interior Color Design Ideation
IRIS: An Interactive Co-pilot for Accelerated Research Ideation 2023 Research Ideation
PersonaFlow: Dynamic Multi-persona Interaction Framework 2024 Research Ideation
VirSci: Autonomous Team-Leader Agent for Virtual Scientific Collaboration 2025 Scientific Research Co-creation
StoryDrawer: Sketch-Based Storyboarding With Multi-agent Discussion 2024 Sketches Generation
ICCRI: Inclusive Co-creative Child-Robot Interaction 2025 Story Generation
Sparkit: Mind-Map-Based Multi-Agent System for Idea Generation Support 2024 Story Generation

Iterative Refinement

Method Year Task Code Link
CharacterMeet: Supporting Creative Writers' Entire Story Character Construction Processes Through Conversation with LLM-Powered Chatbot Avatars 2024 Character Design
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate 2024 Debating (Fairness)
Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate 2024 Hallucination Mitigation
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation 2024 Legal Consultation
ChainBuddy: An AI-assisted Agent System for Generating LLM Pipelines 2025 LLM Pipeline Generation
DesignGPT: Multi-Agent Collaboration in Design 2023 Product Design
Systematic Idea Refinement for Machine Learning Research Agents 2024 Scientific Research Co-creation
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing 2024 Screenwriting
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design 2024 Sketches Generation
Generative Agents: Interactive Simulacra of Human Behavior 2023 Social Simulation

Collaborative Synthesis

Method Year Task Code Link
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks 2024 Agent Benchmarking
AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaboration 2024 Agent Collaboration Visualization
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration 2024 Cognitive Synergy
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models 2024 Creative Writing
Creative Agents: Simulating the Systems Model of Creativity with Generative Agents 2024 Creativity Simulation
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics 2023 Formal Proof
Beyond Code Generation: LLM-supported Exploration of the Program Design Space 2025 Program Design
Agent4Rec: On Generative Agents in Recommendation 2024 Recommendation
How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent 2024 Research Ideation
MARG: Multi-Agent Review Generation for Scientific Papers 2024 Research Peer Review
AgentReview: Exploring Peer Review Dynamics with LLM Agents 2024 Scientific Peer Review
Unlocking AI Creativity: A Multi-Agent Approach with CrewAI 2024 Scientific Research Co-creation
Towards an AI co-scientist 2025 Scientific Research Co-creation
Human--machine co-creation: a complementary cognitive approach to creative character design process using GANs 2024 Silhouette Generation
ChatDev: Communicative Agents for Software Development 2024 Software Engineering
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming 2025 Software Engineering
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis 2024 Story Generation
Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills 2024 Story Generation
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling 2024 Story Generation
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worlds 2024 Story Generation
StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning 2024 Story Generation
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping 2024 UI Prototyping

🌟 Persona and Agent Profile

Coarse-Grained

Method Publication Profiling Method Agent Persona
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration GitHub arXiv(2024) Model-Generated Self-Defined
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play GitHub COLM 2024 Model-Generated Self-Defined
PersonaGym: Evaluating Persona Agents and LLMs GitHub arXiv(2025) Human-defined Self-Defined
Systematic Idea Refinement for Machine Learning Research Agents Tsinghua University Course:AML 2024 Human-Defined Assitant
Sparkit: A mind map-based mas for idea generation support. EMAS 2024 Human-Defined Self-Defined
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate GitHub EMNLP 2024 Human-Defined Debater
An Interactive Co-Pilot for Accelerated Research Ideation HCINLP 2024 Human-Defined Mentor & Colleague
ChainBuddy: An AI-assisted Agent System for Generating LLM Pipelines CHI 2025 Human-Defined Mentor & Planner

Medium-Coarse-Grained

Method Publication Profiling Method Agent Persona
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks GitHub arXiv(2025) Model-Generated Company Employee
HOLLMWOOD: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playin EMNLP 2024 Human-Defined Artist
TRIZ Agents: A Multi-Agent LLM Approach for TRIZ-Based Innovation ICAART 2025 Human-Defined Problem Solver
Towards an AI co-scientist arxiv(2025) Human-Defined Researcher
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming arxiv(2025) Human-Defined Programmer
DesignGPT: Multi-Agent Collaboration in Design ISCID 2023 Human-Defined Self-Defined
CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent GitHub CHI 2024 Human-Defined Researcher
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation arxiv(2024) Human-Defined Lawyer
MARG: Multi-Agent Review Generation for Scientific Papers GitHub arxiv(2024) Human-Defined Expert

Fine-Grained

Method Publication Profiling Method Agent Persona
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas arxiv(2024) Data-Derived Researcher
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System GitHub arxiv(2025) Data-Derived Researcher
On Generative Agents in Recommendation SIGIR 2024 Data-Derived Media User
[PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits] GitHub NAACL 2024 Model-Generated
Self-Defined
The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents arXiv(2025) Model-Generated Self-Defined
Creative Agents: Simulating the Systems Model of Creativity with Generative Agents arXiv(2024) Model-Generated Artist
Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents arXiv(2023) Model-Generated Self-Defined
Generative Agents: Interactive Simulacra of Human Behavior GitHub UIST 2023 Conditional Video Generation Human-defined
AgentReview: Exploring Peer Review Dynamics with LLM Agents GitHub EMNLP 2024 Human-defined Reviewer

🌟 Evaluation

Text Artifact Evaluation

Papers Year Task Subjective Objective Code Link
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking 2025 AUT and RAT TTCT, Boden’s Criterion, and others Semantic similarity
The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents 2025 AUT and others TTCT -
LLMDiscussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play 2024 AUT and others TTCT - Github
An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods 2025 Conceptual Design TTCT -
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models 2024 Idea Generation TTCT -
AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation 2024 Idea Generation Innovation, Insightfullness, and others Semantic Similarity
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation 2024 Legal Consultation Personalization and Professionalism -
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics 2023 Mathematical Proving - BLEU Score Github
DesignGPT: Multi-Agent Collaboration in Design 2023 Product Design Novelty, Completeness, and Feasibility -
Using Generative AI Personas Increases Collective Diversity in Human Ideation 2025 Plot Generation - Semantic Similarity
Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers 2024 Poem Writing Fluency and Creativity -
MARG: Multi-Agent Review Generation for Scientific Papers 2024 Paper Review Generation Specificity and Overall Rating - Github
CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent 2024 Research Ideation Boden’s criterion - Github
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas 2024 Research Ideation Creativity, Usefulness, and Helpfulness -
HOLLMWOOD:Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing 2024 Screenwriting Interestingness, Relevance, and others Entropy-n, Self-BLEU, and others Github
Towards an AI co-scientist 2025 Scientific Research Co-creation Novelty and Impact - Github
Mixed-Initiative Methods for Co-Creation in Scientific Research 2024 Scientific Research Co-creation Novelty, Specificity, and others Semantic Similarity
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System 2025 Scientific Research Co-creation Novelty, Clarity, Feasibility Semantic Euclidean Distance Github
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling 2024 Story Generation Readability, Perceived Creativity, and others - Github
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worlds 2024 Story Generation Interactivity, Coherence, and others Self-BLEU
Creative Wand: A System to Study Effects of Communications in Co-creative Settings 2022 Story Generation Goal completion and Satisfication - Github
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis 2024 Story Generation Creativity Entropy and others Github
Towards Inclusive Co-creative Child-robot Interaction: Can Social Robots Support Neurodivergent Children's Creativity? 2025 Story Generation TTCT -

Image Artifact Evaluation

Papers Year Task Subjective Objective Code Link
C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language Model 2024 Interior Color Design Ideation Inspiring, Reasonableness, and others -
CREA:ACollaborative Multi-Agent Framework for Creative Content Generation with Diffusion Models 2025 Image Editing & Generation Expressiveness, Aesthetic appeal, and others CLIP scores and others
One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor 2025 Meme Generation Funniness, Creativity, and Shareability -
StoryDrawer: A Child–AI Collaborative Drawing System to Support Children's Creative Visual Storytelling 2022 Sketches Generation TTCT FID, TIE ,and Semantic loss
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design 2024 Sketches Generation TTCT - Github
Drawing with Reframer: Emergence and Control in Co-Creative AI 2023 Sketches Generation Novelty and Surprise within MICSI - Github
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANs 2024 Silhouette Generation Designer’s review FID
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping 2024 UI Prototype Generation - FID and Generation Diversity

Interaction Evaluation with User Study

Papers Year Task Code Link
ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation 2024 Image Generation
AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas 2024 Idea Generation
How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent 2024 Research Ideation
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models 2024 Creative Writing

🌟 Dataset

Psychological Test Datasets

Dataset Year Task Data Link

Task Specific Datasets

Paper Task Year Data Link
MacGyver: Are Large Language Models Creative Problem Solvers? Problem-Solving in Physically Grounded Scenarios 2023 Link
Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers Creative Writing 2023 Link
Large Language Models for Human-AI Co-Creation of Robotic Dance Performances Robot Dance Creation 2024 Link
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANs Character Design 2024 Link
TriviaQA Open-ended question task. 2017 Link
Generative Agents: Interactive Simulacra of Human Behavior Open-ended question task. 2023 Link
GPQA: A Graduate-Level Google-Proof Q&A Benchmark Open-ended question task 2023 Link
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models Codenames Task 2022 Link
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics Mathematical Formal Proof Generation and Verification 2023 Link
AI Idea Bench 2025: AI Research Idea Generation Benchmark Idea Generation 2025 Link
LiveIdeaBench: Evaluating LLMs' Divergent Thinking for Scientific Idea Generation with Minimal Context Idea Generation 2024 Link
COMPAS dataset Fairness-Aware Debating 2016 Link

License

This repository is released under the Apache 2.0 license.

About

Survey Paper for Multi-Agent Interaction and Collaboration

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published