SCI-AGENT-EXAM
On SCI-AGENT-EXAM, GPT-6 Astra scores 80.9 overall, with 26.0% of runs exceeding the 0.95 success threshold.
Benchmarking Agents on Scientific Research Tasks
AI Research Radar
Monday, October 26: 4 frontier papers out of 380 read.
Frontier
Papers rated as moving the field
How we chooseRynnUnify
RynnUnify improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano.
Kolibri
“We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes.”
Not an Accident
SIREN finds attacks in all vulnerable SPARK scenarios and causes a real humanoid to collide with a person.
This week in research
Oct 20–26: 308 cleared, 30 frontier
- Security103◆ 1
- Safety56◆ 12
- Agents43◆ 7
- Robotics26◆ 4
- 4 moreScience, Training, Reasoning, ML22◆ 3
- Society18
- Language17◆ 2
- Inference13
- Vision10◆ 1
- Security103◆ 1
- 8 moreSociety, Language, Inference, Vision, Science, Training, Reasoning, ML80◆ 6
- Safety56◆ 12
- Agents43◆ 7
- Robotics26◆ 4
Size: cleared papers◆ and tint: frontier papers
Models under test
Papers this week that name the model
Labs publishing
Papers this week with an author at the lab
Also cleared
30 papers, by field
Security & privacy10
- Fooling LLM Safety Guards into Judging the Wrong Text
- Ulysses: A Security Harness For Synthesizing Verifiable Web Exploits
- M³: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process
- Agentic AI Defense: Decode-Time Provenance Binding for MCP
- TRACE: Data-Free Text Reconstruction Attacks against Approximate Unlearning in LLMs
All 10 papers in Security & privacy
- Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs
- AgentTrace: Execution-Level Fingerprinting of Black-Box LLM Agents
- Detecting Insecure Compositions of Clean Agent Skills via Capability-Flow Graph Analysis
- Watermarking Diffusion Language Models with Learned Foresight and Balanced Bands
- On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection
Agents9
- DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical TasksCode
- SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management
- Incomparable Minima in Agent Tool Authorization
- Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows
- Who Verifies the Tools? How Warped Formatters Mislead Agents and Their Verifiers
All 9 papers in Agents
- DealsFlow: An Evidence-Gated Agentic Harness for Venture Capital Deal Triage
- StateMirror: Pre-Execution Detection of Hallucinated Tool Calls in Coding Agents
- Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations
- Measure Before You Train: Typed Decision Models vs. a Classical Baseline vs. a Generative LLM for Routing a Personal Multilingual Inbox