Thursday, June 11: 4 frontier papers out of 334 read.
Papers rated as moving the field
How we choose“Across frontier model-runtime pairings, the best PassRate reaches only 41.2%, showing the benchmark remains far from saturated.”
The authors, in the abstract
A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
“The headline contrast lives in the hard regime: on SWE-bench Lite, the firewall reduces fabrication from 33.7% (StateFlow) to 0.67%, a paired difference of -33.07 pp [95% CI -36.53, -29.73].”
“In this paper, we demonstrate generalization hacking, in which a model collects reward during RL while preventing the rewarded behavior from generalizing.”
Jun 5–11: 168 cleared, 8 frontier
The week’s papers have not been sorted into fields yet.
Papers this week that name the model
Papers this week with an author at the lab
32 papers