Reward Hacking Challenges Oversight of Autonomous Research Agents
“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
September 19–25: 18 frontier papers out of 2,353 read.
Papers rated as moving the field
How we choose“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
“Experiments demonstrate that PAH enhances the model performance without updating the model parameters or increasing the interaction budget.”
“Evaluations show competitive performance across mathematics, science, programming, general language understanding, and agent tasks, often approaching that of much larger models.”
“The zero spans 790 source sessions, giving a per-session upper bound of 0.38%.”
“Across our evaluations, best-of-3 evasion attempt rates reach up to 98% and success rates up to 88%, with substantial variance across models.”
“Across eleven models from seven architecture families, our evaluation shows 91--100% recovery on nine models with KL divergence 0.001--0.007, while evaluated activation-level detectors remain close to random guessing (AUC <= 0.56).”
“One (GPT-6 Astra) scored 3%, and all others scored 0%.”
“With a reference PoC, the strongest configuration solves 31 of 45 tasks (68.9%).”
“Extensive evaluations demonstrate that Qwen3.8-Omni-Flash achieves strong performance across multimodal understanding, reasoning, long-horizon agentic execution, and video productivity tasks.”
“Starting from Qwen3.5-9B, SCOPE-RL achieves a 54.17% task success rate on OSWorld and a 64.30% attack-avoidance rate on OS-BLIND, yielding the best aggregate capability--safety score of 58.80% among the evaluated agents.”
“Median response length also falls from 19,217 to 18,296 tokens and the truncation rate from 25.8% to 8.3%, so the improvement does not come from thinking longer.”
“We show the suppression can be delivered indirectly and characterize its dependence on the chat template across two more reasoning models, and we evaluate input sanitization, parser hardening, and empty-reasoning detection as defenses; flagging an absent trace catches the basic attack but not an adaptive benign decoy.”
“We show that local LLM agents such as Claude Code, Codex, Antigravity, Open Code and Grok Build fail to enforce this boundary.”
“We show that this assumption breaks down: \emph{individual safety alignment fails to transfer to multi-agent settings}.”
Sep 19–25: 217 cleared, 18 frontier
Papers this week that name the model
Papers this week with an author at the lab
199 papers, by field