AI assists adversarial collaboration in debate on minority salience

Abstract

The advancement of science depends on rigorous tests of competing hypotheses, yet many disputes are left unresolved. Adversarial collaboration—where opposing scientists jointly design decisive tests—is one proposed solution. We examine whether large language models (LLMs) can play a role by organizing information, structuring the debate and generating candidate experimental designs. This article reports an AI-assisted adversarial collaboration designed to resolve a debate in PNAS on minority salience—an overestimation of the percentage of minority faces in a visual display. The debate focused on whether there would be further overestimation when minorities in the displays were the same minorities in participants’ communities (or social environments). Using LLMs to extract and organize competing propositions, we identified central disagreements and generated initial experimental designs to test claims. Human collaborators refined the designs and created two preregistered experiments that factorially manipulated the ethnicity of minority faces and the ethnicity of participants’ communities. Data showed that people exaggerated the percentage of minorities in facial displays. Furthermore, overestimation was even greater when minorities in facial displays were also minorities in participants’ communities. When the two camps of researchers saw the results, their confidence in key hypotheses converged. We do not experimentally test AI-assisted adversarial collaboration relative to traditional adversarial collaboration or other forms of dispute resolution. Rather, our study illustrates how an AI tool can be used with adversarial collaboration to formalize claims, structure disagreements, lower barriers to collaboration, and serve as an impartial observer to strengthen perceptions of fairness.