Continual Learning Bench
“We find that these systems leave headroom for improved continual learning: agents frequently overfit to immediate observations or fail to reuse knowledge across instances, and dedicated memory systems do not fix this -- in fact, naive ICL outperforms systems dedicated to memory management.”
The authors, in the abstract
Evaluating Frontier AI Systems in Real-World Stateful Environments