Generative AI models like LLMs are transforming how we create and access information while also raising concerns about manipulation, deception, and the integrity of public discourse at unprecedented scale.
The AI Manipulation and Information Integrity (AIMII) workshop will bring together researchers from computer science, cognitive science, philosophy, political science, and policy to clarify core concepts, evaluate the evidence on AI's persuasive and manipulative capabilities, and explore implications for society and democracy.
The workshop will feature three panel discussions with leading researchers as well as a poster session showcasing new work from the broader community.
Schedule
| 12:45 - 1:15 |
Lunch |
| 1:15 - 2:00 |
Poster Session |
| 2:00 - 2:05 |
Opening Remarks from Organizing Committee |
| 2:05 - 3:05 |
Panel 1: What is AI manipulation? When and why is it bad?
Carina Prunkl, Elizabeth Edenberg, Gökhan Onel |
| 3:05 - 4:05 |
Panel 2: Measuring manipulative capabilities and behaviors
Maurice Jakesch, Hannah Kirk, Kobi Hackenburg, Jason Hoelscher-Obermaier |
| 4:05 - 4:30 |
Break |
| 4:30 - 5:30 |
Panel 3: Societal impacts and Information Integrity
Dan Williams, Hugo Mercier, Chloé Bakalar, Dino Pedreschi |
| 5:30 - 6:00 |
Closing Remarks & Next Steps |
Accepted Posters
Abstract Submissions
- Tina Austin
UnBlooms for AI Influence Literacy: Measuring Resistance to Model-Generated Persuasion in Learning Contexts
- Mari Cairns
Human Factors and Dark Triad Traits in AI Model Alignment
- Julius Chandler
TruthBot: Parallel Truthful Inference for Detecting and Mitigating AI Manipulation
- Thomas H. Costello, Kellin Pelrine, Matthew Kowal, Antonio A. Arechar, Jean-François Godbout, Adam Gleave, David Rand, Gordon Pennycook
Large language models can effectively convince people to believe conspiracies
- Peter Courtney
Accessible AI and Election Integrity: Societal Risks of AI-Enabled Voter Suppression
- Paul de Font-Reaulx
Capability, Communication, and Character: The 3C Framework for Evaluating Epistemic Performance in Frontier Models
- Kamile Dementaviciute, Guillaume Bied, Tijl De Bie
Measuring Machine Persuasion: A Survey of Automated Evaluation Methods for Large Language Models
- Zhonghao He
Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning
- Haein Kong, A M Muntasir Rahman, Ruixiang Tang, Vivek K. Singh
SafePersuasion: A Dataset, Taxonomy, and Baselines for Analysis of Rational Persuasion and Manipulation
- Linh Le, David Williams-King, Arthur Colle
The Persuasive Power of Personas: Testing AI Policies In The Lab
- Lorelei Logel Demoulin
A practitioners' approach to safe generative AI use in social welfare work
- Jared Moore, Nick Haber, Max Kleiman-Weiner
Understanding How AIs Persuade
- Carlos Mougan, Gokhan Onel
Manipulation and Persuasion under the AI Act
- Tianyi Qiu
Learning to Seek Human Reflective Equilibrium
- Aansh Samyani, Zoe Tzifa-Kratira, Aviel Parrack
Probe Busting: Adversarial Vulnerabilities and Defenses in Deception Detection
- Anish Sathyanarayanan, Aditya Nagarsekar, Aarush Rathore
Chain-of-Thought Manipulation Monitor: Real-Time Detection of Deceptive Reasoning in LLMs
- Semra Sevi, Can Mekik
AI Weakens, But Does Not Strengthen, Political Attitudes
- Syahriza Ilmi Hakim Situmorang, Zhonghao He, Shi Feng, Tianyi Alex Qiu
Can LLMs Simulate Human Belief Change?
- Ekaterina Uetova, Lucy Hederman, Robert Ross, Rasita Vinay, Marily Oppezzo, Dympna O'Sullivan
Do LLM-based Conversational Agents Belong in Online Peer Support Groups? Reflections and Concerns About Their Role in Health Communities
- Feiyi Wang, Dolores Albarracín, Sudeep Bhatia
Integrating Behavioral Science Theory with Large Language Models to Study Persuasion at Scale
- Nadja Winning
Agential Affordances of AI
- Ayse Gizem Yasar
Enforcing the AI Act against harmful manipulation: the case of sycophancy in general-purpose AI models
- Kayo Yin, Jacob Steinhardt
Scaling Laws of Psychological Profiling and Personalized Persuasion in LLMs
- Jocelyn Zhu, Jimin Nam
The Persuasion Perception Gap: Consumer Awareness of LLM Influence
Apart Research Hackathon Winners
- Alex Csaky, Tanzim Chowdhury
Cross-Linguistic Sycophancy in Frontier LLMs: A Benchmark Study
- Ardysatrio Haroen
Eliciting Deception on Generative Search Engines
- Helios Lyons, Horatio Lyons
SycophantSee - Activation-based diagnostics for prompt engineering: monitoring sycophancy at prompt and generation time
- Leonidas Raghav, Choong Kai Zhe
Agent Attacks via Memory Injection
- Jerome Wynne, Nora Petrova
Who Does Your AI Serve? Manipulation By and Of AI Assistants
Topics
We welcome submissions on topics including (but not limited to):
Conceptual & Philosophical Foundations
- Definitions and taxonomies of persuasion, manipulation, and deception
- Moral and epistemic dimensions of AI influence
- Autonomy, consent, and the ethics of personalized persuasion
- Boundary cases and edge cases (e.g. when does influence become manipulation?)
Measurement & Evaluation
- Benchmarks and evaluations of persuasive or manipulative capabilities
- Ecological validity of current measurement approaches
- Sycophancy, reward hacking, and training dynamics that produce manipulative behaviors
- Detecting deception, sandbagging, or strategic behavior in AI systems
- Human studies of AI persuasion (attitude change, belief updating, behavioral effects)
Psychology & Cognitive Science
- Human susceptibility to AI-generated persuasion
- Trust, overreliance, and calibration in human-AI interaction
- Cognitive and affective mechanisms of AI influence
- Individual differences in vulnerability to AI manipulation
Societal & Political Impacts
- AI and misinformation/disinformation
- Effects on journalism, media ecosystems, and information environments
- Implications for democratic deliberation and political discourse
- Manipulation in AI companions, chatbots, and productivity tools
- Targeted advertising, recommender systems, and algorithmic influence
Mitigations & Governance
- Technical approaches to reducing manipulative capabilities or behaviors
- Transparency, disclosure, and labeling interventions
- Regulatory frameworks (EU AI Act, DSA, etc.) and their effectiveness
- Red-teaming, auditing, and third-party evaluation
- Platform governance and content moderation
Broader Perspectives
- Historical and comparative perspectives on information manipulation
- Human-AI co-evolution in communication
- Manipulation in multi-agent and agentic AI systems
- Dual-use concerns and beneficial applications of persuasive AI
Organizers