Reward Hacking Challenges Oversight of Autonomous Research Agents
“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
September 22–28: 19 frontier papers out of 2,411 read.
Papers rated as moving the field
How we choose“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
“Experiments demonstrate that PAH enhances the model performance without updating the model parameters or increasing the interaction budget.”
“Evaluations show competitive performance across mathematics, science, programming, general language understanding, and agent tasks, often approaching that of much larger models.”
“Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B show that even the best model achieves only 52.0% task success, with systematic failure modes including communication breakdowns, role confusion, and inability to maintain shared plans across rounds.”
“Across our evaluations, best-of-3 evasion attempt rates reach up to 98% and success rates up to 88%, with substantial variance across models.”
“Across eleven models from seven architecture families, our evaluation shows 91--100% recovery on nine models with KL divergence 0.001--0.007, while evaluated activation-level detectors remain close to random guessing (AUC <= 0.56).”
“One (GPT-6 Astra) scored 3%, and all others scored 0%.”
“With a reference PoC, the strongest configuration solves 31 of 45 tasks (68.9%).”
“Extensive evaluations demonstrate that Qwen3.8-Omni-Flash achieves strong performance across multimodal understanding, reasoning, long-horizon agentic execution, and video productivity tasks.”
“Starting from Qwen3.5-9B, SCOPE-RL achieves a 54.17% task success rate on OSWorld and a 64.30% attack-avoidance rate on OS-BLIND, yielding the best aggregate capability--safety score of 58.80% among the evaluated agents.”
“Median response length also falls from 19,217 to 18,296 tokens and the truncation rate from 25.8% to 8.3%, so the improvement does not come from thinking longer.”
“We show the suppression can be delivered indirectly and characterize its dependence on the chat template across two more reasoning models, and we evaluate input sanitization, parser hardening, and empty-reasoning detection as defenses; flagging an absent trace catches the basic attack but not an adaptive benign decoy.”
“We show that local LLM agents such as Claude Code, Codex, Antigravity, Open Code and Grok Build fail to enforce this boundary.”
“We show that this assumption breaks down: \emph{individual safety alignment fails to transfer to multi-agent settings}.”
Sep 22–28: 223 cleared, 19 frontier
Papers this week that name the model
Papers this week with an author at the lab
204 papers, by field