June 2–8: 9 frontier papers out of 1,628 read.
Papers rated as moving the field
How we choose“On Lean-IMO-Bench, LEAP boosts the one-shot formal solve rate of general-purpose LLMs from below 10% to 70%, notably surpassing the 48% benchmark set by a specialized, gold-medal-caliber IMO system.”
The authors, in the abstract
Supercharging LLMs for Formal Mathematics with Agentic Frameworks
“Oracle-plan experiments further show that, when plan quality is controlled, the trained duplex model produces high-quality guidance and large gains on Out-of-Plan recovery.”
“Across 61 checkpoint rows and 1920 row-level transitions, aggressive PPO has the highest localized reward-hacking rate (14.45%; bootstrap 95% CI: 10.16-18.75), while UP-PPO yields lower rates in the same aggressive regime (11.33-10.94%).”
“To study this phenomenon, we introduce SocioHack, a sandbox of 72 societal environments, and find that within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery.”
“To evaluate reward-model robustness, we introduce RewardHackBench containing 13 reward-hacking patterns covering real life high-stakes domains and general settings, and we find severe failures on specific subcategories across eight reward models.”
“We find that these systems leave headroom for improved continual learning: agents frequently overfit to immediate observations or fail to reuse knowledge across instances, and dedicated memory systems do not fix this -- in fact, naive ICL outperforms systems dedicated to memory management.”
“Our results demonstrate that self-sustaining AI-driven cyber-threats are no longer theoretical.”
Jun 2–8: 119 cleared, 9 frontier
The week’s papers have not been sorted into fields yet.
Papers this week that name the model
Papers this week with an author at the lab
110 papers