SCI-AGENT-EXAM
On SCI-AGENT-EXAM, GPT-6 Astra scores 80.9 overall, with 26.0% of runs exceeding the 0.95 success threshold.
Benchmarking Agents on Scientific Research Tasks
October 21–27: 18 frontier papers out of 2,463 read.
Papers rated as moving the field
How we chooseOn SCI-AGENT-EXAM, GPT-6 Astra scores 80.9 overall, with 26.0% of runs exceeding the 0.95 success threshold.
Benchmarking Agents on Scientific Research Tasks
RynnUnify improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano.
Claude Opus 5.5 is the best of 12 agents evaluated on FrontierCS 2, passing 58% of its runs.
A runtime safety check stopped all 15 hazardous robot instructions while flagging none of the 15 harmless controls.
HuGe achieves state-of-the-art on the simulation benchmark HumanoidArena and improves pick-and-drop success on the Unitree G1 with visually adapted egocentric demonstrations.
“We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes.”
Spark-Audio 1.0, trained on domestic hardware, matches Qwen3.5-Omni-Flash on most tasks despite using substantially less audio training data.
Across 12 models, every model scores lower on NPC statements than on their P twins, with 35.4% versus 63.8% pass@1.
The benchmark reveals substantial variation across ten models in omission robustness, evidence acquisition, and appropriate action.
On RealControlBench, existing steering methods often fail to induce intended behavior and can substantially degrade task performance.
Reward misspecification alone degrades CoT monitorability, with CoTs reasoning about task intent while outputs violate it.
Ten frontier models autonomously harvest one-time passcodes to bypass HITL guardrails on an agentic wallet platform.
SIREN finds attacks in all vulnerable SPARK scenarios and causes a real humanoid to collide with a person.
Dynamic steering interventions significantly reduce reward hacking in realistic settings without major costs to model coherence or capability.
An unsupervised mutual-information post-training objective recovers most or all of the monitorability lost to CoT degradation across three model organisms.
Oct 21–27: 263 cleared, 18 frontier
Papers this week that name the model
Papers this week with an author at the lab
245 papers, by field