AI agent cooperation Instinct
OpenCollab, AgentSWE and DeepRSI test what agents deliver
Configured teamwork, convincing demos and high development scores all need acceptance checks. OpenCollab audits realized organization, AgentSWE freezes delivered agents for hidden tests, and DeepRSI separates public progress from private results. Longer-horizon systems show why shared state and evaluation budgets matter.
Evaluation
OpenCollab: verify that the configured team actually worked
Hsu and colleagues record role-level events, permissions and resource use, then test whether participation, delegation, role boundaries and information flow match the declared organization. Single-factor changes move adherence from 47.2% to 97.2%. Their two-coder workflow improves DeepSWE Pass@1 from 55.75% to 65.49%.
Assessment
Separate the effect of assigning a team from whether that team materialized at runtime.
Catch-up to version 2, dated October 9 in the fetched primary; paper front matter says September 29.
AgentSWE: the delivered agent must satisfy unseen requirements
Wang and colleagues commission 25 agent-building, editing and optimization tasks. Builders see illustrative development cases; frozen submissions face held-out acceptance cases with requirement-derived rubrics and hard gates. Strongest mean scores are 19.1/100 for Creation and 31.6/100 for Editing; high development scores often fail to transfer.
Assessment
Distinguish a convincing demonstration, an executable runtime and a system that serves the full requirement.
Newly surfaced October preprint.
DeepRSI: a long budget still needs a private acceptance test
Rajaghatta, Devarinti, Berezovsky and Park introduce 15 research-derived optimization problems with 48-hour budgets. Each model runs twice in the Pi harness without network access. Public verifier calls save checkpoints, and private checks grade held-out behavior. Authors document public-verifier overfitting and early stopping that sometimes needs auto-resume.
Assessment
Compare progress curves and private outcomes rather than taking the best visible development score.
October 12 primary announcement, newly surfaced.
Parallel: control data access before crediting the agent architecture
Ethan Kam compares the same agent with open-web access versus added PubMed and ClinicalTrials.gov connectors on 288 LABBench2 questions. Vendor results improve from 254 to 266 correct, reducing error from 11.8% to 7.6%. Traces show connectors were actually used; 15 answers improve and three worsen.
Assessment
A source-access intervention can explain gains that might otherwise be attributed to reasoning or teamwork.
October 13 vendor study.
Coordination research
Stateless Language Agents: keep durable research outside conversations
Zhang and colleagues store candidates and measured outcomes in a harness, giving an Advisor and parallel Workers fresh role-specific context each invocation. On four FrontierSWE tasks, their system trails the best baseline at 25% of budget but leads at full budget. A replicated kernel experiment matches the best baseline with 93.1% fewer tokens.
Assessment
Evaluation horizons can reverse rankings; focused context and explicit experiment assignments are testable design choices.
October 6 preprint, newly surfaced through October 13 coverage.
GitSwarm: persistent contributions are measurable, causal value is harder
Shah and colleagues let asynchronous workers inspect and extend artifacts across Git branches, preserving atomic contributions and semantic dependencies. On a fixed 50-task ProgramBench subset, they report 79.4% mean score versus 65.1% for the strongest reported baseline; 94.7% of contributions are subsequently built upon.
Assessment
Track what later workers consume, including failed experiments, rather than counting messages alone.
Catch-up October preprint, newly surfaced.
Collaborative Theory of Mind: revise before the user has to correct
Zeng and colleagues train an assistant to draft, predict a user reply and revise before responding. With a Qwen3-8B assistant and Qwen3-32B user simulator, collaborative math accuracy improves from 74.1% to 79.6% versus naive GRPO. Prediction supervision reduces simulated user replies on coding and writing under the same reward.
Assessment
Measure the correction burden and final task result together; more clarification questions alone need not help.
October 9 preprint, newly surfaced.
Model slices: train heterogeneous collaborators to hand off reasoning
Wang and colleagues replace Qwen3-14B with 1.7B, 4B and 8B models totaling 13.7B parameters. Protocol distillation and step-level supervision train reasoning and interaction decisions. Across question answering, math, code and constrained generation, authors report comparable or better results than the larger model and compared multi-agent systems.
Assessment
Tests whether smaller distinct models can reassess prior reasoning rather than merely repeat it.
October 9 preprint, newly surfaced.
Infrastructure
Samsung GoAT: retain rejected plans beside the executed path
Samsung researchers describe a graph of agent-task executions, memory retrievals, delegation and rejected candidate plans. Their evaluation uses 2,000 in-house and 1,000 MultiWOZ dialogues, plus a blind explanation study with seven researchers on 50 conversations. The LangGraph implementation is public.
Assessment
Structured provenance can support debugging beyond a chronological replay of tool calls.
Newly surfaced vendor post; exact primary publication date not exposed.
Listen
AI Engineer: Claude for long-horizon tasks
Lance Martin discusses durable session context, separate-context verification, in-band memory and out-of-band consolidation in a 25-minute talk. Publisher transcript and chapters cover external context and verification around 5:20, continual-learning evaluation around 10:39 and memory correction around 13:51.
Assessment
Worth hearing alongside the research on persistent shared state: preservation and verification solve different problems.
Catch-up talk; publication day unknown.
Tina Huang: expert-panel takeaways on evals and workflows
This 23-minute panel recap discusses vertical agents, prompting, evaluation and workflow decomposition. Aggregator chapters place core skills around 9:11, writing evals around 13:12 and workflow decomposition around 19:28.
Assessment
A lighter practical listen on turning an agent idea into a testable workflow.
Original video published June 23, 2025; a third-party audio adaptation listed it on October 13, 2026.
Editorial synthesis
A team diagram is a hypothesis about behavior, not proof that collaboration happened. The strongest evaluations record actual participation, freeze artifacts for unseen acceptance tests, and separate accessible feedback from final verification. Reuse traces and explanations make work inspectable, but neither establishes causal benefit. Report quality alongside realized organization, budget and horizon, including the cost of checking the result.
Editorial interpretation of the cited sources.