July 1–7: 15 frontier papers out of 2,035 read.
Papers rated as moving the field
How we choose“The strongest and the most cost effective agent, Codex GPT-5.5, achieves only approximately 42% success rate.”
The authors, in the abstract
A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents
“Even the strongest frontier system in its default reasoning mode reaches only about 59.3% on Domain Q increasing thinking depth within a tier does not move performance materially, while moving to a higher product tier yields modest gains.”
“Across five model pairs spanning four model families, we find that 91.3% of reasoning-mode gains are knowledge-dependent rather than structure-invariant (63/69 gains; Wilson 95% CI [82.3%, 96.0%]), directly challenging the assumption that chain-of-thought reasoning improves short-horizon procedural scientific problem-solving.”
“The same stream also quantifies the failure modes: a loose guard silently mislabels 48.9% of compiled answers, and an unfaithful deopt reference causes the verification gate to refuse recompilation.”
“An adversarial evaluation of monitors of increasing semantic depth shows false-allows falling from 75% (name-based gating) to zero (dynamic effect tracing), and refusal history transfers compliance to held-out red-line families.”
“Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces.”
“All ASR claims are 3-judge-calibrated (cell rankings stable across judges; absolute levels \pm18pp; see App. B.13).”
“Human-as-Humanoid yields a 4.8--7.2x raw demonstration-throughput gain over humanoid teleoperation in our data-collection analysis, and on several downstream tasks, policies post-trained only with the converted human labels generalize to real-robot deployment without target-task robot demonstrations.”
“Extensive evaluations show that MMBench-Live preserves stable model rankings, maintains semantic alignment with the original benchmark, and exhibits weaker contamination-related memorization signals, suggesting a practical and scalable paradigm for sustainable multimodal benchmark evolution.”
“Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts.”
Jul 1–7: 128 cleared, 15 frontier
The week’s papers have not been sorted into fields yet.
Papers this week that name the model
Papers this week with an author at the lab
113 papers