Paper Detail

PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios

Ali Zamani, Soroush Esfahanian, Erfan Shahabi, Mohammad Taher Pilehvar, Azadeh Shakery, Yadollah Yaghoobzadeh

arxiv Score 14.8

Published 2026-10-14 · First seen 2026-10-15

General AI

Abstract

Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play, yet prior benchmarks typically evaluate them in isolation using synthetic scenarios. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions. The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{zamani2026perso,
  title = {PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios},
  author = {Ali Zamani and Soroush Esfahanian and Erfan Shahabi and Mohammad Taher Pilehvar and Azadeh Shakery and Yadollah Yaghoobzadeh},
  year = {2026},
  abstract = {Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play},
  url = {https://arxiv.org/abs/2610.20548},
  keywords = {cs.CL, cs.AI, cs.SI},
  eprint = {2610.20548},
  archiveprefix = {arXiv},
}

Metadata

{}