Multi-Reward Hacking
Trust-Anchored Gating achieves 0.581 pass@4 versus 0.220 for additive aggregation on Qwen3-4B with a brevity reward.
From Aggregation-Induced Drift to Trust-Anchored Defense
October 13–19: 26 frontier papers out of 2,856 read.
Papers rated as moving the field
How we chooseTrust-Anchored Gating achieves 0.581 pass@4 versus 0.220 for additive aggregation on Qwen3-4B with a brevity reward.
From Aggregation-Induced Drift to Trust-Anchored Defense
“We demonstrate these benefits in RL environments where reward penalties cause models to learn to reward hack in undetected ways, training models ranging from 135M to 8B parameters.”
“Our experiments on 8 robot configurations reveal substantial gaps: 1) robot demonstrations required for ICL are expensive to collect, and 2) it is implausible to trade longer execution for higher success rates with physical constraints.”
“Across all evaluated systems and backbones, no system passes more than 1.5% of forget requests end to end, and most pass none at all.”
“GPT-5.6-Sol scores 23.2% on unassisted whole-problem solving against 28.8% and 30.4% under the two sequential metrics, a gap measuring the integration burden.”
The 2.4B REKI model sets a Pareto frontier between performance and size, outperformed only by models about twice its size.
“On AgentDojo, an external prompt-injection benchmark, the same readout calibrated on 29 tasks, with the steering unchanged, cuts attack success on unseen tasks in two families (Gemma-4 11.7% -> 5.7%, gpt-oss 28.5% -> 17.5%) while keeping benign utility (Gemma-4 0.637 vs 0.637; gpt-oss 0.667 vs 0.681).”
Recursive self-distillation of safe models on benign cues progressively misaligns them on harmful queries across Qwen3 and Phi-4.
Fine-tuned model weights covertly exfiltrate secrets via DNS from sandboxed LLM agents, firing in 20/20 adversarial suppression trials.
On PhysClosureBench, Claude-Opus-5 achieves the highest mean Atomic Score (6.29/10) but reaches RC4 in only 32.7% of attempts.
“Experiments on three Qwen1.5 scales show that checkpoints with prior harmful exposure generally retain greater hazardous concept accessibility after the same safety alignment, with the strongest separation in intermediate and later layers.”
“On the physical robot, six models with end-effector control refused 18% of 612 trials against 19% in sim, and three with joint control refused 41% of 252 against 31%.”
“Thinking is a flag of the chat template rather than a separate model, and lifts mathematical reasoning by up to 20 points on the 8B model.”
“Across four models and two in-domain, three held-out corpora, only ASC lowers the error rate of authored sentences and writes more supported ones in every setting, by 38% and 46% at the median under held-out evaluators, giving readers more grounded information.”
“Further analysis shows that later requests with deeper or more numerous dependencies are solved less accurately, while runtime errors become more frequent over successive requests.”
“We show an attack technique that allows to alter the watermarking message.”
“Evaluation results demonstrate substantial improvements over previous LLM-jp models and competitive performance against open-weight models of similar scale.”
“We show that frontier models can receive full credit for responses with little substantive answer content.”
Oct 13–19: 304 cleared, 26 frontier
Papers this week that name the model
Papers this week with an author at the lab
278 papers, by field