Paper Detail
Eunkyu Park, Markelle Roesti, Wesley Hanwen Deng, Renata Barreto, Mohammad Tahaei, Kenneth Holstein, Jason Hong, Motahhare Eslami
AI auditing increasingly incorporates AI agents to expand the scale and breadth of audit coverage, yet little is known about how auditing work should be divided without displacing human judgment. We introduce Human-Agent Audit Collaboration (HAAC), a workflow and system for structuring human-AI collaboration in AI auditing. Drawing on prior work and formative consultations with AI auditing practitioners, HAAC specifies how agents can support exploration, assessment, reporting, and review while preserving human oversight where contextual judgment is critical. We instantiate HAAC for conversational shopping agents and evaluate it through two studies. With 71 auditors, AI assistance increased attack success and broadened exploration, while also shaping later attacks and increasing auditors' reliance on AI-generated assessments and reports. Interviews with Responsible AI practitioners showed that actionable audits require visibility into coverage, reproducible attack trajectories, and evaluation of the auditing agents themselves. Our findings identify design considerations for effective and accountable human-AI auditing.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@article{park2026who,
title = {Who Does What in AI Auditing? Designing Human-AI Collaboration for Auditing Generative AI},
author = {Eunkyu Park and Markelle Roesti and Wesley Hanwen Deng and Renata Barreto and Mohammad Tahaei and Kenneth Holstein and Jason Hong and Motahhare Eslami},
year = {2026},
abstract = {AI auditing increasingly incorporates AI agents to expand the scale and breadth of audit coverage, yet little is known about how auditing work should be divided without displacing human judgment. We introduce Human-Agent Audit Collaboration (HAAC), a workflow and system for structuring human-AI collaboration in AI auditing. Drawing on prior work and formative consultations with AI auditing practitioners, HAAC specifies how agents can support exploration, assessment, reporting, and review while p},
url = {https://arxiv.org/abs/2609.24986},
keywords = {cs.HC},
eprint = {2609.24986},
archiveprefix = {arXiv},
}
{}