WebArena-Pro
On WebArena-Pro, Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 both reach 62.3%, the best among evaluated agents.
A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
Monday, November 16: 4 frontier papers out of 270 read.
Papers rated as moving the field
How we chooseOn WebArena-Pro, Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 both reach 62.3%, the best among evaluated agents.
A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
Soteria achieves 1.7 times the average task success rate of the best baselines across tool-use benchmarks, with no violations of policy specifications.
An invisible activation edit overwrites reasoning models' beliefs about their own requests, causing confabulated defenses of injected topics despite warnings.
Nov 10–16: 278 cleared, 18 frontier
Papers this week that name the model
Papers this week with an author at the lab
31 papers, by field