Reward Hacking Challenges Oversight of Autonomous Research Agents
“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
September 23–29: 23 frontier papers out of 3,173 read.
Papers rated as moving the field
How we choose“An LLM panel reviewing only submitted code and reported scores misses 33/505 confirmed hacks (6.5%).”
The authors, in the abstract
“We compare the performance of a cascading system to open-weight and closed SLMs on MULTISPEECH-BENCH and find that the cascading system outperforms open-weight SLMs but not all closed SLMs.”
“Our extensive experiments demonstrate that RMB significantly improves reward accuracy on both in-distribution and out-of-distribution datasets, substantially mitigating the reward hacking issue and ultimately improving RLHF performance.”
“Experiments demonstrate that PAH enhances the model performance without updating the model parameters or increasing the interaction budget.”
“Evaluations show competitive performance across mathematics, science, programming, general language understanding, and agent tasks, often approaching that of much larger models.”
“Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B show that even the best model achieves only 52.0% task success, with systematic failure modes including communication breakdowns, role confusion, and inability to maintain shared plans across rounds.”
“Experiments across multiple Code-as-Policy embodied tasks show that RE-0 improves both teacher-assisted execution and the standalone student, and generalizes to novel robots and scenes.”
“In AgentDojo, it reduces non-DoS targeted attack success from 20.82% to 0.40%.”
“Across our evaluations, best-of-3 evasion attempt rates reach up to 98% and success rates up to 88%, with substantial variance across models.”
“Across eleven models from seven architecture families, our evaluation shows 91--100% recovery on nine models with KL divergence 0.001--0.007, while evaluated activation-level detectors remain close to random guessing (AUC <= 0.56).”
“One (GPT-6 Astra) scored 3%, and all others scored 0%.”
“With a reference PoC, the strongest configuration solves 31 of 45 tasks (68.9%).”
“Extensive evaluations demonstrate that Qwen3.8-Omni-Flash achieves strong performance across multimodal understanding, reasoning, long-horizon agentic execution, and video productivity tasks.”
“We show the suppression can be delivered indirectly and characterize its dependence on the chat template across two more reasoning models, and we evaluate input sanitization, parser hardening, and empty-reasoning detection as defenses; flagging an absent trace catches the basic attack but not an adaptive benign decoy.”
“We find that model pairs can learn to communicate the secret using only one bit of feedback indicating whether the receiver inferred it correctly.”
“We show that simple attacks can steal reasoning capabilities from existing closed-source language models using data easily obtainable from current APIs, yielding reasoning improvements equivalent to more sophisticated attacks that extract the full hidden traces.”
“We show that local LLM agents such as Claude Code, Codex, Antigravity, Open Code and Grok Build fail to enforce this boundary.”
“We show that this assumption breaks down: \emph{individual safety alignment fails to transfer to multi-agent settings}.”
Sep 23–29: 270 cleared, 23 frontier
Papers this week that name the model
Papers this week with an author at the lab
247 papers, by field