AI agent cooperation Instinct
Catch-up: group memory, refutation checks, cultural adaptation
A Sunday catch-up edition: a real-world group-chat benchmark finds memory systems recall but fail to use, and a refutation-first verifier beats confirmation checks. Plus cultural norm assimilation over repeated interactions, curiosity in agent teams, and a trust-responsive authorization policy.
- group-memory
- verification-refutation
- cultural-adaptation
- agent-authorization
- curiosity
- catch-up
Evaluation
- Authors’ claims
- Catch-up item
A group assistant that recalls everything still fails to use it
Hu, Zhang, Zhao, Zhu and colleagues introduce GroupAssistBench, the first memory benchmark built from real-world group conversations with a deployed group assistant: 80,299 messages in 50 groups spanning trip planning, classes, hobby communities and project coordination, with 2,395 evaluation questions sampled from 9.5K validated candidates at a construction cost over $53K. Two orthogonal axes separate memory recall (L1, questions with reference answers) from memory use (L2, open service requests scored by instance-tailored rubrics whose penalties derive from real user feedback), crossed with three memory dependency types; 56% of M2 and 93% of M3 instances require memories anchored in at least two members' messages. Across 14 configurations covering four backbones with full context, retrieval methods and external memory systems, the strongest memory system beats same-backbone Full Context on recall (0.621 versus 0.570) yet falls far behind on use (0.247 versus 0.431), and a paired analysis of 497 memories per configuration shows recalled memories are often retrieved but unused at L2. Two additions that relate retrieved memories to each other and bind remembered facts as response constraints bring consistent gains.
Assessment
The recall-versus-use split is the gap most memory benchmarks flatten into a single score, and the cross-member, conflict-tracking dependency types are what real group assistance actually requires. Construction details worth borrowing: requests kept only when group history changes the answer, memories anchored to verbatim spans, rubric penalties derived from real user feedback.
Catch-up: submitted November 3; announced in the November 4 listing.
- Authors’ claims
- Catch-up item
Verify actions by trying to refute them, not to confirm them
Oh, Heo, Noh, Kim, Son and Ok argue that pre-execution verification has a directional blind spot: a wrong action often passes every check that asks whether it is right. Their BlackSwan procedure runs expect, suspect, test: the agent predicts each state-mutating action's intended effect, preregisters specific and empirically testable suspicions for how it could be wrong, then probes them using existing observations or additional non-mutating actions, executing only if no suspicion survives testing. On the most challenging AppWorld tasks, BlackSwan improves success rate over the strongest baseline by 14.9 and 25.2 percentage points with and without the backbone model's thinking mode respectively, and even without thinking or in-context demonstrations it outperforms every baseline equipped with thinking by 11.7 points. The gains transfer beyond AppWorld, with WebShop performance improving by 12.7 points.
Assessment
Refutation-first checking is an inference-time complement to policy guardrails: instead of proving an action allowed, the agent must fail to prove it wrong. The headline comparison (direction of verification beats amount of reasoning) is directly relevant to how delegation safeguards get evaluated.
Catch-up: submitted November 2; announced in the November 3 listing.
- Authors’ claims
- Catch-up item
Multi-turn RL wins need room to run, and can reverse without it
Zhu, Jin, Tu, Men and colleagues evaluate multi-turn agentic reinforcement learning across sampling breadth K and interaction depth T via Pass@(K,T), and show that trajectory-level RL advantages depend on sufficient turn budgets, diminishing significantly when budgets tighten and even reversing on certain tasks; the drop is not explained by tasks becoming structurally unsolvable, since a structural performance upper bound per task stays reachable. Comparing base and RL model actions under identical states, they find agentic RL does not uniformly elevate single-step decision quality: its strength concentrates in error recovery and long-horizon resilience, which partly explains why RL models need a larger budget to show their advantage. To mitigate the deployment risk, they discuss benefits and limits of process supervision signals and introduce BudgetMix RL, which varies interaction budgets during training and achieves robust performance across diverse execution settings.
Assessment
A budget-sensitive metric for a budget-sensitive claim: Pass@(K,T) keeps the turn allowance inside the measurement instead of letting trajectory scores silently assume unlimited interaction. The step-quality finding warns that higher task success does not certify better individual actions, which matters most where actions are irreversible.
Catch-up: submitted November 4; announced in the November 5 listing.
Coordination research
- Authors’ claims
- Catch-up item
Cultural norms are learned across episodes, not in one prompt
Liu, Reichart and Korhonen reframe cultural adaptation as a continuous process of norm assimilation through social interaction, moving evaluation from what models know to how they adapt. In their lightweight sandbox environment, an agent holds dialogue sessions with NPCs under scenario-specific cultural norms while a judge model assigns a social reputation score after each session, and that score serves as the learning signal. Across repeated episodes they find that memory mechanisms foster longitudinal learning: reflective-memory configurations raise judge-scored norm compliance over no-memory variants across five country scenarios (China, France, Japan, New Zealand, Saudi Arabia), the adaptation transfers to unfamiliar scenarios and to synthetic norms, and norm assimilation works across five languages. A validation pass with Gemma NPCs finds dialogue leakage in 8.3% of cases.
Assessment
Cooperation competence is longitudinal: this work operationalizes norm learning as a measurable trajectory (reputation over episodes) rather than a one-shot quiz, a shape that group-interaction benchmarks can adopt. The transfer and synthetic-norm results separate genuine assimilation from scenario memorization.
Catch-up: submitted November 5; announced in the November 6 listing.
- Authors’ claims
- Catch-up item
Curious agent teams beat incurious ones on seven of eight tasks
Borah, Bhatti, Jin and Mihalcea introduce CuriousTogether, a framework evaluating curiosity-driven multi-agent LLM interactions across three complementary dimensions: novelty-seeking, epistemic and social curiosity. Across eight tasks spanning reasoning, creative generation and social bias mitigation, curiosity-guided multi-agent settings improve over non-curious ones on seven of eight tasks, with gains up to 20.6% in story generation (semantic diversity) and a 14.9 percentage point reduction in stereotype rates. Country-conditioned cultural grounding benefits story generation and social tasks but can hurt precision-oriented reasoning, and the ethics task is the one case where cultural grounding beats curiosity. The gains are inference-time, with favorable quality-to-token trade-offs against token-matched single-agent baselines.
Assessment
Induced curiosity works like exploration pressure inside an agent team, and the ablation design (all-curious, one-curious, no-curious) shows the effect does not require every agent to participate. The task-dependent cost of cultural grounding is a useful warning against blanket persona prompting in multi-agent setups.
Catch-up: submitted October 31; announced in the November 3 listing.
Infrastructure
- Authors’ claims
- Catch-up item
Slow to trust, fast to restrict: an authorization policy measured
Shihab, Felde, Akhmedov, Gubo and colleagues empirically characterize the Trust Responsive Authorization Policy (TRAP), which maps an Authorization Readiness Score, a weighted combination of runtime trust, operational stability and contextual risk, onto discrete authorization levels through an asymmetric rule: escalation requires consecutive observations above a band edge plus a margin, while a sharp drop in readiness restricts authority at once. On one matched set of 100 Monte Carlo traces they compare TRAP with static, trust-only, risk-only and symmetric policies. Under abrupt degradation, TRAP restricts at onset and re-escalates sooner after recovery than the symmetric policy, with the highest and, descriptively, least variable stability among the adaptive policies, although more of its post-recovery escalations occur below the selected trust threshold; paired bootstrap intervals exclude zero for the remaining differences. One-factor sweeps show recovery time rises with the escalation margin and run length and shifts with the score weights, but is unchanged by the restriction threshold.
Assessment
The asymmetric shape (slow to escalate, fast to restrict) is the sensible default for runtime agent authorization, and the paper's contribution is treating the policy itself as a measurable artifact: paired traces, bootstrap intervals and sensitivity sweeps instead of anecdote. That evaluation template transfers to any runtime guard that meters an agent's authority.
Catch-up: submitted November 4; announced in the November 5 listing.
Also seen
- Authors’ claims
- Catch-up item
ScamWorld grades web agents on the fraud that targets humans
Wu, Liu, Wen, Huang, Wang and Ma introduce ScamWorld, an interactive environment that evaluates how web agents respond during task execution to fraud designed to deceive humans, including fraudulent online stores, fake investment platforms and advance-fee schemes, working around the short lifespan of real scam sites that has blocked systematic evaluation.
Assessment
Turns agent safety around: not what the agent does to users, but what the world does to the agent. A repeatable fraud environment fills a real gap left by short-lived scam sites.
Catch-up: submitted November 3; announced in the November 4 listing.
- Authors’ claims
- Catch-up item
Latent agent swarms that share KV caches instead of text
Jiang and Karniadakis present SwarmEvolve, a framework in which hundreds of low-cost agents per round work in swarms, reason in latent space without decoding to text, and share information by passing KV caches, while a novelty filter keeps the most distinct ideas by directly measuring distances, aiming to cut the cost of LLM-guided evolutionary search.
Assessment
KV-cache sharing as a communication channel is an interesting infrastructure idea for cheap swarm coordination, though the contribution is evolutionary search, not cooperation research.
Catch-up: submitted November 5; announced in the November 6 listing.
- Authors’ claims
- Catch-up item
Distributed neural populations grow shared conventions
Toquebiau, Botoko Ekila, Van Eecke and Beuls train populations of bi-directional neural agents in a fully distributed fashion on referential games, arguing that prior emergent-communication setups (speaker-listener splits, two-agent populations, shared internal representations) prevent studying language emergence through cultural evolution, and show the approach scales to large populations. A revised version surfaced in this week's listings; thematically adjacent to the zero-shot convention work covered on November 5.
Assessment
The distributed-population training setup pairs naturally with yesterday's cultural-adaptation sandbox: both treat convention formation as a multi-episode process rather than a single game.
Catch-up: v1 submitted August 24; v2 appeared in the November 3 listing.
Editorial synthesis
The weekend's backlog splits along a use axis. Group assistants can recall what a group said yet fail to act on it, and agents can verify what supports an action yet miss what refutes it; in both cases the capability measured is not the capability deployed. The coordination items push on time: cultural norms and curiosity gains both emerge across repeated interaction, not inside single episodes, which is where most evaluations still live.
Editorial interpretation of the cited sources.