AI agent cooperation Instinct
When agents should stop, what audits miss, where facts vanish
Passing tests do not settle when an agent should stop. Clinical benchmark audits find the largest hindsight effect in case text, not simulated labs. Two shared-memory studies show why collecting more facts need not preserve disagreement.
- anytime evaluation
- clinical hindsight
- correction utility
- conflict-preserving memory
- evidence handoffs
Evaluation
- Authors’ claims
Done, but still working: evaluate the patches before the final one
Yang and colleagues report an anytime evaluation of about 6,000 coding-agent runs spanning eight open models and two scaffolds on SWE-bench Verified. Replaying commands and scoring intermediate patches permits paired comparisons of stopping or earlier-patch selection against finishing the same run, without new trajectories when replay preserves the relevant states. The hindsight cost saving available at passing self-tests ranges from under 1% to 57% across model and scaffold settings. For Qwen3.6-27B in mini-SWE-agent, stopping after two passes on an unchanged patch saves 22.8% of inference cost at a 1.0-point loss in local resolution in a preregistered replication of 386 runs. The uncertainty-based test does not establish non-inferiority at the 2-point margin. The paper decomposes quality loss into rule firing, later patch changes and net repair per change.

Assessment
Intermediate-state evaluation distinguishes wasted work from work that repairs a patch. Paired replay is especially useful for evaluating stopping policies, but it is not a replacement for fresh runs when an intervention changes the agent's behavior.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
HindAudit locates the biggest hindsight effect in the case text
Zhu and colleagues report HindAudit, separating three channels in clinical-agent benchmarks: the diagnosis supplied to a simulator, retrospective simulator context, and agent-facing case text. On a cohort of 27,489 MIMIC-IV emergency admissions, five registered tests did not detect diagnosis leakage into simulated laboratory results; giving the diagnosis to five simulators had negligible effects, while prompted gatekeepers diluted real diagnostic signal. The larger effect was in case construction: across four LLMs, benchmark-style summaries yielded 39.7% zero-shot accuracy and discharge histories 45.3%, versus 25.4% from a presentation-time reconstruction based on triage records. The initial case-text analysis was exploratory, with the gap reproduced in a registered held-out-patient replication. Six interactive agents also scored higher from benchmark text, while their ranking changed little.

Assessment
An audit can target the wrong channel. The useful contribution is the separation of case construction from simulator behavior, and absolute score inflation from changes in model ranking.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
- Catch-up item
Consistency is not correction: check what the gate can change
Sethi reports an exact decomposition of correction utility into gated recoveries minus gated harms, separating answer ranking, selective prediction and improvement after intervention. For a correct replacement-only gate, the initial error count bounds the beneficial discordant pairs it can produce. In the paper's 60-answer diagnostic set, four initial errors are reachable while six one-directional changes are needed for the paired exact test to reject at 0.05, leaving even a perfect gate unable to produce a significant improvement. In a small-model MiniHouse suite, a perturbation-agreement verifier reaches within-policy AUROC 0.990 yet changes no episode outcomes relative to single-sample decoding at 4.32 times the call count. In the controlled 840-episode study, none of the verifier-driven policies replaces the initial proposal on the 60 GSM8K decisions.

Assessment
A confidence signal can be excellent at ranking and useless as a controller. Reporting gate firing, reachable errors, harms and recoveries makes a null result interpretable before more evaluation budget is spent.
Catch-up: submitted October 15; first encountered in the November 18 listing.
Coordination research
- Authors’ claims
Naming the bad teammate is easier than recovering around it
Heo and colleagues report 720 episodes in a custom four-player Overcooked environment, crossing three maps, four teammate conditions and two monitoring prompts, with 30 trials per cell. The main experiment uses gpt-5.4-mini throughout and injects one fixed-policy misaligned teammate: speech-action mismatch, role invasion or aggregate-progress disruption. Eventual verbal attribution reaches 90%, 88% and 97% respectively in the weak-sensitivity correction analysis of 90 episodes per troll condition (30 per map), but an utterance combining attribution with situational diagnosis appears in only 61%, 54% and 31%. Recovery is clearest for the speech-action mismatch and weakest for aggregate disruption. Stronger attribution prompting improves the aggregate-disruption condition from 2.7 to 4.8 soups, while reducing troll-free performance from 8.0 to 5.0 soups, a 37.5% drop.

Assessment
Robustness needs separate measures for detection, recovery and false-alarm cost. The design makes that tradeoff visible instead of treating more explicit blame as an unconditional improvement.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
DebateGraph: preserve disagreement before the debate sees it
Khan and colleagues report a 39-run gated ablation of five shared-memory mechanisms for multi-agent debate. In the graph backend at retrieval cap 200, surfacing the ranked conflict set instead of collapsing it to one winning fact improves answer token-F1 by 0.340, with a 95% interval of 0.267 to 0.414, and reduces successful attacks from 22 to 7 of 122 poisoned items under the same prompt budget, using 11,318 tokens per query for collapse and 11,550 for surface. The pre-declared cap of 40 had hidden most planted contradictions from retrieval and measured a smaller effect: 0.043 F1 overall and 0.084 on poisoned items. At cap 200, the write trust-gate and router provenance features instead cost 0.075 and 0.096 F1 in their respective contrasts. Surfacing helps on a second retrieval substrate, with a positive HotpotQA accuracy effect on its pre-specified discriminating subset; the full-sample interval includes zero.

Assessment
The storage layer can destroy precisely the evidence a debate is meant to adjudicate. The retrieval-cap audit is as important as the positive result: an apparently weak mechanism may never have had access to its target.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
Delegation retrieves more conflicts but hands fewer of them through
Zhang and colleagues report controlled numerical-conflict injection into 11 Harvey LAB document-review tasks, comparing no, optional and required delegation across GPT-5.6 Luna, GPT-6 Luna and Claude Haiku 5.5, with two runs per model. Pooled over 438 conflict observations per policy at comparable generation spending, required delegation increases retrieval of both conflicting claims from 50.2% to 68.7% relative to no delegation, while judged conflict detection falls from 16.2% to 13.2%. Under required delegation, only 35.2% of pairs retrieved by workers survive together in worker reports; pair recall at lead evidence is 24.4%. Matched comparisons associate keeping both claims within one worker context with better detection, and splitting them across workers with worse detection, relative to single-agent execution.

Assessment
Evidence collection and evidence preservation are different capabilities. Profiling source reads, worker reports, lead evidence and final reports exposes losses a task-level pass rate would conceal.
Submitted November 17; appeared in the November 18 listing.
Infrastructure
- Authors’ claims
GPT-6.2 release puts long-horizon agent cost on the comparison sheet
OpenAI reports the release of GPT-6.2 Astra and Sol. On its announcement's Agents' Last Exam comparison, Astra scores 5.2 points above GPT-6 Astra at less than one-third the cost. It also reports lower-token long-horizon workflows and model availability through its API and Work/Codex products. The announcement states that the comparator results for Claude Fable 5.2 exclude nine tasks that used a fallback model.
Assessment
A relevant new baseline for agent evaluation, particularly when task quality and aggregate inference cost are reported together. This is a model-release claim, not evidence of improved multi-agent cooperation.
Announced November 17; new since the previous daily lab-blog check.
Also seen
- Authors’ claims
Recovered capacity stays idle unless agents can spread demand
Ezaki and colleagues report simulated shared-server choices in which agents must discover that a server has recovered despite a stale status report. For Claude Haiku 5.5, a rule reminder prompts no agent to try the recovered server, while a successful-request record prompts all to choose it. All five tested models still leave capacity unused after evidence sharing. With GPT-6.1 Sol and Gemini 3.5 Flash-Lite, distinct numerical cues spread simultaneous requests and increase completions without much change in overall choice frequencies.
Assessment
Availability knowledge does not solve congestion: coordination also needs agents to distribute demand.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
Look-ahead-free trading agents fail to beat the simple bets
Wang reports a look-ahead-free comparison before 598 earnings announcements and after 409 bad-news drops in large US stocks. Across five models from four vendors, staged agents do not beat simple buy-all-event baselines. Ticker-and-date-only probes show a GPT-based model predicting announcement direction more accurately before its training cutoff than after it.
Assessment
Event-agent evaluations need simple-action baselines and identity-blind, post-cutoff tests before apparent alpha is credited to reasoning.
Submitted November 17; appeared in the November 18 listing.
- Authors’ claims
Federated trust infrastructure for cross-organization agents
Nguyen and Husain propose Agent Trust Infrastructure, combining organizational certificate authorities, zero-trust checks and remote attestation as an external enforcement layer. They report a simulated eight-organization federation spanning six frameworks and five-agent teams. The security experiments use two organizations, 14 attack scenarios and six benign tasks; across the 11 scenarios comparable under every configuration, containment rises from 40% to 80% with all enforcement mechanisms enabled.
Assessment
An architecture proposal focused on the identity and configuration checks needed when agents cross organizational boundaries.
Submitted November 17; appeared in the November 18 listing.
Listen
Listen: who measures trust when agents trade without a broker?
Obadia and Haimes discuss the coordination layer needed for agents to negotiate and transact across organizations without a central intermediary. Obadia explains ARIA's planned Arena as an environment for testing whether trust infrastructure survives adversarial pressure. Haimes challenges profit as the core score and asks whether it makes safety secondary; Obadia describes separately measuring security and leaves room to revise the metric. They also discuss responsibility when an agent breaks a deal. The episode runs 1:40:49; publisher transcript entry points are 34:39 for the Arena, 40:29 for profit versus safety, and 54:38 for liability.
Assessment
The most useful part is the disagreement about what a coordination benchmark should reward. It connects evaluation design to incentives, negotiation and accountability rather than treating successful communication as the outcome.
Catch-up: published November 9; not covered in the November 9 through 17 reports. Added November 18 during recovery of omitted listening.
Listen: long-running agents need observable outcomes, not only traces
Holdstock-Brown and Bronsdon discuss the infrastructure for agents that retain state over months: durable execution, sub-agent orchestration, sandboxes and the growing API load generated by agents. Holdstock-Brown argues for evaluating production behavior using outcome events, such as whether a review passed or a change was merged, alongside conventional agent evaluations. About 53 minutes. Publisher-linked segments cover agent-driven API usage at 9:12, load compounding at 13:16, and outcome-event monitoring at 15:52.
Assessment
A practical complement to benchmark papers: it asks what a deployed system can observe cheaply and continuously, and where an outcome proxy is more useful than another model judging a whole trajectory.
Catch-up: published November 12; not covered in the November 12 through 17 reports. Added November 18 during recovery of omitted listening.
Editorial synthesis
A final score hides the route to it. Today's evaluations trace intermediate patches, separate hindsight channels, and count whether a proposed correction can ever change enough outcomes to be measured. The coordination studies make a related distinction: finding a problem is different from preserving its evidence and reorganizing around it. More monitoring or more workers can make the system worse when those handoffs are not measured.
Editorial interpretation of the cited sources.