Multi-Reward Hacking
Trust-Anchored Gating achieves 0.581 pass@4 versus 0.220 for additive aggregation on Qwen3-4B with a brevity reward.
From Aggregation-Induced Drift to Trust-Anchored Defense
October 10–16: 26 frontier papers out of 2,859 read.
Papers rated as moving the field
How we chooseTrust-Anchored Gating achieves 0.581 pass@4 versus 0.220 for additive aggregation on Qwen3-4B with a brevity reward.
From Aggregation-Induced Drift to Trust-Anchored Defense
“We demonstrate these benefits in RL environments where reward penalties cause models to learn to reward hack in undetected ways, training models ranging from 135M to 8B parameters.”
“Using post-RL agents from the mid-training setting as sources, we observe propagation through code and shared memory, and across roles through natural-language plans.”
“Across all evaluated systems and backbones, no system passes more than 1.5% of forget requests end to end, and most pass none at all.”
“GPT-5.6-Sol scores 23.2% on unassisted whole-problem solving against 28.8% and 30.4% under the two sequential metrics, a gap measuring the integration burden.”
The 2.4B REKI model sets a Pareto frontier between performance and size, outperformed only by models about twice its size.
“On AgentDojo, an external prompt-injection benchmark, the same readout calibrated on 29 tasks, with the steering unchanged, cuts attack success on unseen tasks in two families (Gemma-4 11.7% -> 5.7%, gpt-oss 28.5% -> 17.5%) while keeping benign utility (Gemma-4 0.637 vs 0.637; gpt-oss 0.667 vs 0.681).”
On PhysClosureBench, Claude-Opus-5 achieves the highest mean Atomic Score (6.29/10) but reaches RC4 in only 32.7% of attempts.
“Experiments on three Qwen1.5 scales show that checkpoints with prior harmful exposure generally retain greater hazardous concept accessibility after the same safety alignment, with the strongest separation in intermediate and later layers.”
“On the physical robot, six models with end-effector control refused 18% of 612 trials against 19% in sim, and three with joint control refused 41% of 252 against 31%.”
“Thinking is a flag of the chat template rather than a separate model, and lifts mathematical reasoning by up to 20 points on the 8B model.”
“Across four models and two in-domain, three held-out corpora, only ASC lowers the error rate of authored sentences and writes more supported ones in every setting, by 38% and 46% at the median under held-out evaluators, giving readers more grounded information.”
“Further analysis shows that later requests with deeper or more numerous dependencies are solved less accurately, while runtime errors become more frequent over successive requests.”
“We show an attack technique that allows to alter the watermarking message.”
“Evaluation results demonstrate substantial improvements over previous LLM-jp models and competitive performance against open-weight models of similar scale.”
“We show that frontier models can receive full credit for responses with little substantive answer content.”
“We show this assumption does not hold under adversarial conditions by constructing backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger.”
Oct 10–16: 286 cleared, 26 frontier
Papers this week that name the model
Papers this week with an author at the lab
260 papers, by field