README.md

SocialAgent

A collection of resources that investigate social agents.

📣 Update

• [2024/12]: We released the initial version of our survey about social simulation driven by LLM-based Agents: From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents
• [2024/12]: We released the slides of the tutorial in LMG2024.

📖 Contents

• Tutorial Materials
• Paper List

📕 Tutorial

Part	Link
Part I: Introduction	slides
Part II: Individual Simulation	slides
Part III: Scenario Simulation	slides
Part IV: Society Simulation	slides

🔥 Paper List

• Individual Simulation
  • Characters
  • Demographics
• Senario Simulation
  • Dialog-Driven
    • Social Interaction
    • Question Answering
    • Game
  • Task-Driven
    • Foundational and Applied Science
    • Software Development
    • Other Industries
• Society Simulation
  • General Economic
    • Game Theory and Strategic Interactions
    • Economic Contexts
  • Sociology and Political Science
    • Public Opinion Survey
  • Individual and Organizational Behavior Observation
  • Online Platform
    • Social Platforms
    • Recommendation Environments

Individual Simulation

Characters

• [16 Dec 2024] Zhou et al., CharacterBench: Benchmarking Character Customization of Large Language Models
• [9 Dec 2024] Wang et al., Simulating Human-like Daily Activities with Desire-driven Autonomy
• [7 Dec 2024] Wang et al., CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds
• [4 Dec 2024] Kim et al., LLM-Mirror: A Generated-Persona Approach for Survey Pre-Testing
• [18 Nov 2024] Xu et al., Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?
• [1 Mar 2024] Yu et al., Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
• [16 Feb 2024] Shen et al., RoleEval: A Bilingual Role Evaluation Benchmark for Large Language Models
• [9 Jan 2024] Tu et al., CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation
• [14 Dec 2023] Shao et al., Character-LLM: A Trainable Agent for Role-Playing
• [28 Nov 2023] Zhou et al., CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models
• [15 Oct 2024] Ran et al., Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data
• [8 Oct 2024] Yuan et al., Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works
• [7 Jun 2024] Wang et al., InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews
• [18 Jun 2024] Wang et al., RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
• [9 Oct 2023] Chen et al., Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters
• [18 Aug 2023] Li et al., ChatHaruhi: Reviving Anime Character in Reality via Large Language Model
• [6 Aug 2023] Park et al., Generative Agents: Interactive Simulacra of Human Behavior
• [July 2023] Agrawal et al., Multimodal Persona Based Generation of Comic Dialogs
• [9 May 2023] Schwitzgebel et al., Creating a Large Language Model of a Philosopher
• [12 Sep 2021] Brahman et al., "Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding

Demographics

• [3 Dec 2024] Li et al., CultureLLM: Incorporating Cultural Differences into Large Language Models
• [1 Nov 2024] Xie et al., Can Large Language Model Agents Simulate Human Trust Behavior?
• [4 Oct 2024] Huang et al., Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench
• [25 Sep 2024] Abbasian et al., Conversational Health Agents: A Personalized LLM-Powered Agent Framework
• [24 Sep 2024] Ge et al., Scaling Synthetic Data Creation with 1,000,000,000 Personas
• [24 Aug 2024] Chen et al., Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena
• [28 Feb 2024] Sun et al., Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information
• [15 Feb 2024] Weng et al., ControlLM: Crafting Diverse Personalities for Language Models
• [8 Feb 2024] Lee et al., Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Bias
• [29 Oct 2023] Jiang et al., Evaluating and Inducing Personality in Pre-trained Language Models
• [28 Oct 2023] Xiang et al., Language Models Meet World Models: Embodied Experiences Enhance Language Models
• [22 Oct 2023] Chawla et al., Be Selfish, But Wisely: Investigating the Impact of Agent Personality in Mixed-Motive Human-Agent Interactions
• [16 Oct 2023] Shea et al., Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning
• [21 Sep 2023] Serapio-García et al., Personality Traits in Large Language Models
• [30 Jul 2023] Pan et al., Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models
• [29 May 2023] Cheng et al., Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
• [24 May 2023] Song et al., Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs
• [10 May 2023] Karra et al., Estimating the Personality of White-Box Language Models
• [28 Apr 2023] Xie et al., The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges
• [11 Apr 2023] Deshpande et al., Toxicity in ChatGPT: Analyzing Persona-assigned Language Models
• [18 Jan 2023] Horton et al., Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
• [14 Sep 2022] Argyle et al., Out of One, Many: Using Language Models to Simulate Human Samples
• [23 Aug 2022] Liu et al., Improving Personality Consistency in Conversation by Persona Extending

Scenario Simulation

Dialog-Driven

Social Interaction

• [25 Oct 2024] Mou et al., AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
• [3 Aug 2024] Zhang et al., Self-Emotion Blended Dialogue Generation in Social Simulation Agents
• [1 May 2024] Yan et al., Social Life Simulation for Non-Cognitive Skills Learning
• [5 Apr 2024] Yang et al., Social Skill Training with Large Language Models
• [4 Apr 2024] Ataei et al., Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation
• [22 Mar 2024] Zhou et al., SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents

Question Answering

• [9 Oct 2024] Liang et al., Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
• [5 Sep 2024] Ma et al., Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models
• [4 Jun 2024] Xia et al., Measuring Bargaining Abilities of LLMs: A Benchmark and a Buyer-Enhancement Method
• [18 Oct 2023] Xiong et al., Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
• [3 Oct 2023] Wu et al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
• [14 Aug 2023] Chan et al., ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
• [23 May 2023] Du et al., Improving Factuality and Reasoning in Language Models through Multiagent Debate
• [17 May 2023] Fu et al., Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback
• [24 Apr 2023] Hao et al., ChatLLM Network: More Brains, More Intelligence

Game

• [13 Oct 2024] Lan et al., LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay
• [13 Sep 2024] Chen et al., S-Agents: Self-organizing Agents in Open-ended Environments
• [29 Aug 2024] Du et al., Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Game
• [24 Jul 2024] Chi et al., AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
• [2 Jul 2024] Sreedhar et al., Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs
• [17 Jun 2024] Zhu et al., PLAYER: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games*
• [27 May 2024] Zhang et al., Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
• [11 May 2024] Xu et al., Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
• [29 Mar 2024] Wu et al., Enhance Reasoning for Large Language Models in the Game Werewolf
• [29 Feb 2024] Wu et al., Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games
• [20 Feb 2024] Xu et al., Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game
• [29 Dec 2023] Shi et al., Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game
• [8 Nov 2023] Light et al., AvalonBench: Evaluating LLMs Playing the Game of Avalon
• [24 Oct 2023] Wang et al., Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation
• [1 Jun 2023] Zhu et al., Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory

Task-Driven

Foundational and Applied Science

• [23 Dec 2024] Yu et al., ResearchTown: Simulator of Human Research Community
• [8 Nov 2024] Zhu et al., AutoTQA: Towards Autonomous Tabular Question Answering through Multi-Agent Large Language Models
• [21 Aug 2024] Xie et al., DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework
• [11 Jun 2024] Yukhymenko et al., A Synthetic Dataset for Personal Attribute Inference
• [24 May 2024] Li et al., CulturePark: Boosting Cross-cultural Understanding in Large Language Models
• [5 May 2024] Li et al., Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents
• [11 Apr 2024] Baek et al., ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
• [20 Feb 2024] Jin et al., What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents
• [16 Feb 2024] Gao et al., Reviewer2: Optimizing Review Generation Through Prompt Generation
• [8 Jan 2024] D'Arcy et al., MARG: Multi-Agent Review Generation for Scientific Papers
• [16 Nov 2023] Tang et al., MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
• [10 Nov 2023] Zheng et al., ChatGPT Research Group for Optimizing the Crystallinity of MOFs and COFs
• [18 Jul 2023] Wu et al., Large Language Models Perform Diagnostic Reasoning
• [23 May 2023] Hassan et al., ChatGPT as your Personal Data Scientist

Software Development

• [8 Apr 2024] Zhang et al., AutoCodeRover: Autonomous Program Improvement
• [7 May 2024] Qian et al., Iterative Experience Refinement of Software-Developing Agents
• [28 Dec 2023] Qian et al., Experiential Co-Learning of Software-Developing Agents
• [16 Jul 2023] Qian et al., ChatDev: Communicative Agents for Software Development
• [15 Apr 2023] Dong et al., Self-collaboration Code Generation via ChatGPT

Other Industries

• [18 Dec 2024] Xu et al., TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
• [23 Nov 2024] Liu et al., The Decoy Dilemma in Online Medical Information Evaluation: A Comparative Study of Credibility Assessments by LLM and Human Judges
• [5 Sep 2024] Yu et al., From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents
• [23 Jul 2024] Sun et al., LawLuo: A Chinese Law Firm Co-run by LLM Agents
• [26 Jun 2024] Baker et al., Simulating The U.S. Senate: An LLM-Driven Agent Approach to Modeling Legislative Behavior and Bipartisanship
• [23 May 2024] Zhang et al., Towards efficient llm grounding for embodied multi-agent collaboration.
• [5 Mar 2024] He et al., SimuCourt: Building Judicial Decision-Making Agents with Real-world Judgement Documents
• [25 Jan 2024] Tan et al., True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning
• [11 Jun 2024] Qian et al., Scaling Large-Language-Model-based Multi-Agent Collaboration
• [16 Oct 2023] Xie et al., OpenAgents: An Open Platform for Language Agents in the Wild
• [29 Sep 2023] Chen et al., AutoAgents: A Framework for Automatic Agent Generation
• [27 Sep 2023] Chen et al., Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?
• [24 Sep 2023] Weiss et al., Rethinking the Buyer’s Inspection Paradox in Information Markets with Language Agents
• [7 Sep 2023] Li et al., TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance
• [21 Aug 2023] Chen et al., AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
• [10 Jul 2023] Zhao et al., RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
• [5 Jul 2023] Zhang et al., Building Cooperative Embodied Agents Modularly with Large Language Models
• [5 Jun 2023] Talebirad et al., Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
• [27 May 2023] Lin et al., SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
• [31 Mar 2023] Li et al., CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
• [12 Jan 2023] Hamilton et al., Blind Judgement: Agent-Based Supreme Court Modelling With GPT

Social Simulation

General Economic

Game Theory and Strategic Interactions

• [28 Aug 2024] Han et al., Static network structure cannot stabilize cooperation among Large Language Model agents
• [28 Aug 2024] Mensfelt et al., Logic-Enhanced Language Model Agents for Trustworthy Social Simulations
• [19 Jun 2024] Fontana et al., Nicer than humans: How do large language models behave in the prisoner’s dilemma?
• [5 Jun 2024] Mozikov et al., The Good, the Bad, and the Hulk-like GPT: Analyzing Emotional Decisions of Large Language Models in Cooperation and Bargaining Games
• [8 May 2024] Noh et al., LLMs with Personalities in Multi-issue Negotiation Games
• [19 Feb 2024] Wu et al., Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
• [7 Feb 2024] Xie et al., Can large language model agents simulate human trust behaviors?
• [3 Jan 2024] Guo et al., Economics Arena for Large Language Models
• [28 Nov 2023] Hua et al., War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars
• [26 Oct 2023] Zhao et al., CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents
• [21 Aug 2023] Han et al., "Guinea Pig Trials" Utilizing GPT: A Novel Smart Agent-Based Modeling Approach for Studying Firm Competition and Collusion

Economic Contexts

• [21 Nov 2024] Dizaji ,Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model
• [18 Oct 2024] Ji et al., SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent
• [14 Sep 2024] Chopra et al., On the limits of agency in agent-based models
• [8 Mar 2024] Zhou et al., Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs
• [16 Oct 2023] Li et al., EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities
• [20 Sep 2023] Ghaffarzadegan et al., Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence
• [11 Jul 2023] Williams et al., Epidemic Modeling with Generative Agents
• [18 Jan 2023] Horton et al., Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?

Sociology and Political Science

Public Opinion Survey

• [6 Dec 2024] Ju et al., Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models
• [15 Nov 2024] Park et al., Generative Agent Simulations of 1,000 People
• [28 Oct 2024] Zhang et al., ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents
• [6 May 2024] Chaudhary et al., Large Language Models as Instruments of Power: New Regimes of Autonomous Manipulation and Control
• [28 Feb 2024] Sun et al., Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information
• [12 Nov 2023] Xiao et al., Simulating Public Administration Crisis: A Novel Generative Agent-Based Simulation System to Lower Technology Barriers in Social Science Research
• [1 Nov 2023] Lee et al., Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Bias
• [14 Sep 2022] Argyle et al., Out of One, Many: Using Language Models to Simulate Human Samples

Individual and Organizational Behavior Observation

• [10 Dec 2024] Li et al., Political Actor Agent: Simulating Legislative System for Roll Call Votes Prediction with Large Language Models
• [20 Nov 2024] Copollilo et al., Engagement-Driven Content Generation with Large Language Models
• [5 Nov 2024] Takata et al., Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communities
• [18 Oct 2024] Tang et al., Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation
• [5 Aug 2024] Zhu et al., Generative Organizational Behavior Simulation using Large Language Model based Autonomous Agents: A Holacracy Perspective
• [25 Jun 2024] Chuang et al., Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks
• [18 Jun 2024] Jin et al., AgentReview: Exploring Peer Review Dynamics with LLM Agents
• [13 Mar 2024] Ren et al., Emergence of Social Norms in Generative Agent Societies: Principles and Architecture
• [3 Feb 2024] Yu et al., Affordable Generative Agents
• [16 Nov 2023] Chuang et al., The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents
• [16 Nov 2023] Chuang et al., Simulating Opinion Dynamics with Networks of LLM-based Agents
• [8 Nov 2023] Jarrett et al., Language Agents as Digital Representatives in Collective Decision-Making
• [1 Nov 2023] Chuang et al., Simulating Public Opinion about Global Warming: An Empirical Assessment of Algorithmic Fidelity and Bias
• [3 Oct 2023] Kaiya et al., Lyfe Agents: Generative agents for low-cost real-time social interactions
• [3 Oct 2023] Zhang et al., Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
• [3 Oct 2023] Suzuki et al., An evolutionary model of personality traits related to cooperative behavior using a large language model
• [2 Oct 2023] Chuang et al., ChoiceMates: Supporting Unfamiliar Online Decision-Making with Multi-Agent Conversational Interactions
• [7 Aug 2023] Li et al., Quantifying the Impact of Large Language Models on Collective Opinion Dynamics
• [7 Apr 2023] Park et al., Generative Agents: Interactive Simulacra of Human Behavior
• [18 Aug 2022] Aher et al., Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies

Online Platform

Social Platforms

• [14 Dec 2024] Zhang et al., TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System
• [29 Nov 2024] Jin et al., Build An Influential Bot In Social Media Simulations With Large Language Models
• [25 Nov 2024] Ferraro et al., Agent-Based Modelling Meets Generative AI in Social Network Simulations
• [18 Nov 2024] Yang et al., OASIS: Open Agent Social Interaction Simulations with One Million Agents
• [2 Nov 2024] Zhang et al., A Large-scale Time-aware Agents Simulation for Influencer Selection in Digital Advertising Campaigns
• [24 Oct 2024] Liu et al., From a Tiny Slip to a Giant Leap: An LLM-Based Simulation for Fake News Evolution
• [28 Sep 2024] Wang et al., Decoding Echo Chambers: LLM-Powered Simulations Revealing Polarization in Social Networks
• [1 Aug 2024] Rossetti et al., Y Social: an LLM-powered Social Media Digital Twin
• [7 Jul 2024] Xu et al., MINDECHO: Role-Playing Language Agents for Key Opinion Leaders
• [5 May 2024] Cai et al., Language Evolution for Evading Social Media Regulation via LLM-based Multi-agent Simulation
• [14 Mar 2024] Liu et al., From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News
• [27 Feb 2024] Touzel et al., A Simulation System Towards Solving Societal-Scale Manipulation
• [26 Feb 2024] Mou et al., Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation
• [5 Oct 2023] Tornberg et al., Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms
• [18 Aug 2022] Park et al., Social Simulacra: Creating Populated Prototypes for Social Computing Systems
• [7 Aug 2023] Ren et al., BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
• [27 Jul 2023] Gao et al., S3: Social-network Simulation System with Large Language Model-Empowered Agents
• [7 Apr 2023] Park et al., Social Simulacra: Creating Populated Prototypes for Social Computing Systems

Recommendation Environments

• [12 Dec 2024] Liu et al., LMAgent: A Large-scale Multimodal Agents Society for Multi-user Simulation
• [28 Feb 2024] Zhang et al., Prospect Personalized Recommendation on Large Language Model-based Agent Platform
• [16 Oct 2023] Zhang et al., On Generative Agents in Recommendation
• [13 Oct 2023] Zhang et al., AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
• [31 Aug 2023] Huang et al., Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
• [5 Jun 2023] Wang et al., RecAgent: A Novel Simulation Paradigm for Recommender Systems