S-EMBER
A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
July 7–13: 11 frontier papers out of 1,939 read.
Papers rated as moving the field
How we chooseA Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
“The same stream also quantifies the failure modes: a loose guard silently mislabels 48.9% of compiled answers, and an unfaithful deopt reference causes the verification gate to refuse recompilation.”
“Scaling reduces but does not eliminate hacking: even the 32B model retains a 54.9% worse rate under outcome-only rewards, whereas answer-aware rewards improve the oracle trend at every scale.”
“An adversarial evaluation of monitors of increasing semantic depth shows false-allows falling from 75% (name-based gating) to zero (dynamic effect tracing), and refusal history transfers compliance to held-out red-line families.”
“Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces.”
“Extensive experimental analysis reveals substantial performance disparities across evaluation protocols, demonstrating that navigation success alone is an insufficient predictor of real-world application effectiveness.”
“We show it fails structurally: conditioned on a candidate, a judge scores plausibility, not correctness, leaving false-positive basins a policy learns to exploit.”
Jul 7–13: 148 cleared, 11 frontier
The week’s papers have not been sorted into fields yet.
Papers this week that name the model
Papers this week with an author at the lab
137 papers