Report for

Choose a day

AI agent cooperation From the report of

One malicious delegate, replayed incidents and the cost of compaction

  • Authors’ claims

Peer review does not narrow AI analyst disagreement; exemplars do

Gao and Xiao report an agentic multiverse analysis of empirical research. In a pilot, 150 autonomous coding agents (100 Sonnet, 50 Opus) analyzing the same NYSE TAQ data disagree through the same defensible methodological choices human analysts make, though over a narrower range. AI peer review leaves their dispersion almost unchanged, while exposure to top-rated exemplar reports collapses it, largely through imitation rather than analysis. The full framework has agents enumerate a published paper's fork space, score each fork against a validity rubric, execute the validated grid and cross-examine one another under strict separation of duties. Applied to ten management-journal replication packages (1,755 specification estimates), it reproduces every paper's original estimates and separates conclusions that hold across all validated specifications from ones sensitive to individual choices the published text does not discuss.

Adapted from a source figure Figure 3, complete six-panel chart with legend, caption excluded as a crop adaptation; proportions preserved and whitespace padded to 1200x600.Full resolution
Effect-size distributions across six hypotheses by stage: S1 independent analysis, S2 after peer review, S3 after exemplar exposure. Peer review leaves dispersion nearly unchanged; exemplar exposure shifts or narrows it unevenly across hypotheses.
Adapted from a source figure: Ruijiang Gao and Steven Chong Xiao, When AI Analysts Disagree, arXiv 2611.25198, Figure 3, complete six-panel chart with legend, caption excluded as a crop adaptation; proportions preserved and whitespace padded to 1200x600.. License: arXiv nonexclusive distribution license; reproduced as a quotation accompanying this review with attribution

Assessment

The pilot is a clean warning about agent ensembles: having agents review each other does not buy diversity, and showing them strong examples buys conformity, not rigor. Separation of duties, where no agent checks its own work, is the part worth copying into multi-agent research pipelines.

Submitted November 18; appeared in the November 20 listing.