AI agent cooperation Instinct
Phantom consensus, hidden norms and self-organizing swarms
Monday's listing is unusually rich: a benchmark catches LLM groups flipping to a consensus nobody privately held, and NormGym shows agents barely learn norms from others. Plus self-organizing swarms, parallel agent teams, and a survey of what people will delegate.
- phantom-consensus
- normative-competence
- swarm-organization
- parallel-teams
- delegation
- reversibility
Evaluation
- Authors’ claims
Groups flip to a consensus nobody privately held
Sheshanarayana, Pal, Sinha and Dasgupta name and measure a pluralistic-ignorance-style failure in LLM collectives: agents shift their public decisions after observing ambiguous social signals while misreporting what other agents privately judged, even when the group's recorded private judgments do not support the apparent consensus. Their PHANTOM-Bench separates isolated private judgments, public decisions after ambiguous signals, elicited estimates of the group's private judgment, transparency-based recovery, and controls for neutral public text and explicit vote-following across five belief-context conditions, with metrics including private-public gap, collective rationality failure rate, and a transparency recovery rate. Across nine closed- and open-weight models, ambiguous public signals consistently degrade decision quality: public accuracy drops by as much as 47.8 percentage points (Qwen-2.5-7B falls from 77.0% to 29.2%; Qwen-2.5-32B from 73.1% to 42.1%), and collective rationality failures reach 47.3% and 43.3% for those two models in the most susceptible setting, turning correct private group judgments into incorrect public collective outcomes.
Assessment
Consensus aggregation is the default robustness argument for multi-agent setups; this benchmark shows the aggregation step itself can manufacture agreement that no member holds. The private, public, perceived-group separation is directly reusable instrumentation for any evaluation that asks a group of agents to deliberate.
Submitted November 6; announced in the November 9 listing.
- Authors’ claims
Agents obey the norm they were punished for, not the one they saw
Li, Zhang, Baek, Hadfield and Shu introduce NormGym, an interactive benchmark in which a newcomer agent must infer a hidden, arbitrary norm while pursuing its own task inside a multi-agent community, distinguishing what is possible from what is permitted and transferring the inferred norm to unseen layouts. Episodes run up to 100 steps across three social groups with 300 test episodes, and each is constructed so that a shortest-path, norm-blind policy necessarily violates the norm. Normative competence remains hard: the strongest evaluated LLM reaches a joint task-and-compliance success rate of 55.3%. The behavioral analysis shows agents can revise their behavior after sanctions for their own violations but struggle to infer the norm from other agents' violations or compliant behavior, and fine-tuning yields little transfer, although training agents to predict permissible actions for both themselves and others improves competence somewhat.
Assessment
This is the acquisition side of norm competence, complementary to yesterday's cultural-adaptation sandbox: not recognizing a known norm but inferring an unknown one from social evidence. The own-violation versus peer-evidence asymmetry is a crisp, measurable failure mode for any agent joining an existing community.
Submitted November 5; announced in the November 9 listing.
Coordination research
- Authors’ claims
Swarms that learn when to talk beat fixed organization
Jha, Faria, Li, Geng, Brahman, Smith, Tsvetkov and Kleiman-Weiner ask whether agents can learn to organize themselves instead of following a static, hand-designed harness. Their s3 method is, in their framing, the first multi-agent reinforcement learning approach in which agents decide for themselves how long to work alone and when to share findings or read teammates' messages, with no predefined organization or roles. The learned swarms take different approaches, read each other's work, and exhibit emergent complementary roles that generalize to bigger teams. With matched training and inference budgets, s3 swarms evaluated with twice as many agents as in training solve over 25 more coordination problems than the strongest baseline, and in parallel exploration, communicating agents reduce repeated work by nearly 50% and outperform best-of-N by up to 9.4 percentage points.
Assessment
Most multi-agent orchestration encodes the organization chart in the prompt or the harness; this moves the who-talks-to-whom decision into learned policy. The two settings studied (combining private information versus efficient parallel search) are exactly the two coordination modes a cooperation benchmark should score separately.
Submitted November 5; announced in the November 9 listing.
- Authors’ claims
Many small teams beat one big team at the same budget
Wang, Chen, Li, Chalamala and Liu compare three ways to spend test-time compute on agents when a verifier can score every attempt: best-of-N over independent agents, one large collaborating team, and their method, parallel agent teams, which samples many small teams and keeps the best team's result. On 12 long-horizon tasks spanning GPU kernels, formal proofs and exploit development, with a 27B model and a budget of 100 agents, parallel teams of ten beat parallel agents on 10 of 12 tasks and beat one team of 100 on all four tasks where such large teams were run; at equal dollar cost the parallel teams beat a frontier model on half the tasks. A simple statistical model explains the result: individual agents lose because their many samples have a low mean, while one large team wastes the variance that best-of selection needs. Inside a team, sharing discoveries helps and adding a manager hurts.
Assessment
A practical orchestration answer with a mechanism attached: team structure is a mean-versus-variance trade, and intermediate team sizes win. The negative manager result is a useful counterweight to hierarchical multi-agent designs.
- Wang, Chen, Li, Chalamala and Liu, Test-Time Compute Scales Better with Parallel Agent Teams (arXiv) —
Submitted November 6; announced in the November 9 listing.
- Authors’ claims
What 289 people will hand to an agent, and what they keep
Wheeler, Rammouz, Qian and colleagues report a scenario-based survey of 289 U.S. adults with prior AI experience covering 45 use cases across nine life domains, with each respondent evaluating one scenario per domain and repeated measures of automation preference, trust, collaboration, AI leadership, privacy concern and willingness to pay. Automation preferences varied substantially across domains. Their generalized estimating equation models associate greater automation preference with greater trust in the AI agent, desired AI leadership, prior task-specific AI use and anticipated financial savings, while concerns that automation would reduce personal fulfillment predict lower preference.
Assessment
Delegation is a social contract before it is a capability question, and this gives the contract's current shape: domain-level variation with interpretable predictors rather than a single acceptability score. Useful grounding for deciding which tasks an agent should offer to take versus ask about.
Submitted November 6; announced in the November 9 listing.
Infrastructure
- Authors’ claims
Keep the off switch working while the collective learns
Han and Boyle propose Reversible Multi-agent Systems as a research agenda: collectives that grow while an authorized supervisor can still restrict specified effects, keep them restricted as agents adapt, and reconstitute useful cooperation under renewed authority. Their motivating observation is that the mechanisms that make a collective capable, such as rules that route information to agents that act on it, are natural restriction points, yet as agents learn, share memory, delegate and repair failures, the collective can find other paths to the same effect: the original control stays operable but stops being effective while routine performance and compliance checks still pass. What must be reversed, in their framing, is the ability to produce a specified effect, and restrictions must hold even if learned knowledge persists, so the goal is to preserve effective intervention rather than any particular mechanism.
Assessment
This names the governance hole in self-improving multi-agent systems: control points erode as the system re-routes around them, silently. Framing restrictability as a maintained property of the collective, verified during growth rather than assumed at design time, is the right shape for runtime authorization work like TRAP to grow into.
Submitted November 6; announced in the November 9 listing.
Also seen
- Authors’ claims
Multi-agent debate weaponized against fact-checking
Huang, Liang, Lei, Ma and Ojha present Debate-to-Deceive, a multi-agent debate framework grounded in human debate structure and rhetorical style that generates adversarial passages to flip LLM fact-checking judgments, formalizing the adversarial process as an evolutionary game in which agents adapt their strategies. The same debate machinery that improves reasoning is used here to defeat it.
Assessment
A safety-relevant mirror of the debate literature: if debate improves accuracy, a game-theoretic debate adversary is the stress test that fact-checking pipelines will face.
Submitted November 5; announced in the November 9 listing.
- Authors’ claims
Turning debate into training signal without gold answers
Verma, Wang, Rezaei, Rabinovich and Chen convert multi-agent debate into training supervision for reasoning post-training where gold answers, verifiers and reward models are scarce, arguing that majority voting lacks principled tie-breaking and is ill-defined for open-ended tasks while homogeneous committees reinforce correlated errors.
Assessment
Self-supervised supervision from debate is a promising route when verifiers are scarce; the tie-breaking critique of majority voting is well taken and testable.
Submitted November 6; announced in the November 9 listing.
- Authors’ claims
Models trust the guru over the scientist on unverifiable claims
Guan runs three behavioral studies across six models on how far demonstrated reliability travels when a new claim cannot be checked. Adapting the Einstein-effect paradigm, the models, unlike the human comparison sample, favor a spiritual leader over a scientist for the same obscure statement, and the preference largely reverses when the statement is reworded in the vocabulary of physics.
Assessment
Credibility transfer is a trust-formation primitive for agent societies: how models generalize who-to-believe shapes every information cascade downstream.
Submitted November 6; announced in the November 9 listing.
Editorial synthesis
Two measurement results land on the same nerve: groups are not automatically wiser than their members. Ambiguous chatter can flip an LLM collective to an answer nobody privately holds, and agents learn norms from their own punishments while ignoring everyone else's. The constructive items push back: swarms that learn when to talk, and many small teams instead of one big one, both show organization itself is learnable rather than designed.
Editorial interpretation of the cited sources.