June 11–17: 19 frontier papers out of 1,945 read.
Papers rated as moving the field
How we choose“Across frontier model-runtime pairings, the best PassRate reaches only 41.2%, showing the benchmark remains far from saturated.”
The authors, in the abstract
A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
“Nemotron 3 Ultra achieves up to 6x higher inference throughput as compared to state-of-the-art publicly available LLMs while attaining on-par accuracy.”
“We show that reinforcement learning can make a policy \emph{addicted} to such a visible self-benefit channel.”
“Evaluating 15 advanced agents reveals that robust spatial task solving remains challenging: the strongest model, GPT-5, achieves an average task success rate (TSR) of only 17.4%, while the leading open-source model, Qwen-3.5, reaches 14.1%.”
“GPT-5.5 with Codex performs best, solving 16.1% of web exploitation tasks and 31.7% of post-exploitation tasks; with more concrete hints, these rates increase to 33.0% and 46.3%.”
“We demonstrate that the attack amplifies privacy leakage across language (up to 100% successful extraction), and vision-language models (up 90% successful extraction).”
“The strongest adaptive search (tree-of-attacks) breaks Opus 4.8 on 11.5% of intents overall, whereas Fable 5 stays in the single digits (6.1% worst-case).”
“We evaluate 19 open-weight and proprietary LLMs, and find that current models achieve penetration success rates ranging from 10.7% to 69.3%.”
“The headline contrast lives in the hard regime: on SWE-bench Lite, the firewall reduces fabrication from 33.7% (StateFlow) to 0.67%, a paired difference of -33.07 pp [95% CI -36.53, -29.73].”
“Experiments show that GeoDisaster challenges existing RS-VLMs and agentic systems, while RCEA improves tool use, evidence grounding, state consistency, and decision generation.”
“Our experiments using a custom decomposer show that state-of-the-art agents exhibit high refusal rates on monolithic harmful tasks, but significantly lower refusal rates on their decomposed variants, while often inadvertently fulfilling the adversarial objectives.”
“Our results show that reward hacking arises naturally when optimizing proxy objectives with capable language model agents and resists standard mitigations, suggesting that proxy-reward failures in agentic settings may require approaches beyond standard exploration and credit-assignment fixes.”
“Extensive experiments demonstrate that our method achieves surgical concept erasure performance across various domains with minimal training overhead, timely and elegently resolute the critical safety vulnerabilities inherent in emerging VAR architectures.”
“Our results reveal a critical vulnerability in post-training: alignment can be overridden by a single example.”
“In this paper, we demonstrate generalization hacking, in which a model collects reward during RL while preventing the rewarded behavior from generalizing.”
Jun 11–17: 204 cleared, 19 frontier
The week’s papers have not been sorted into fields yet.
Papers this week that name the model
Papers this week with an author at the lab
185 papers