SCI-AGENT-EXAM
On SCI-AGENT-EXAM, GPT-6 Astra scores 80.9 overall, with 26.0% of runs exceeding the 0.95 success threshold.
Benchmarking Agents on Scientific Research Tasks
AI Research Radar
October 24–30: 15 frontier papers out of 2,351 read.
Frontier
Papers rated as moving the field
How we chooseSpaceAct
Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1, with seven solving at most 27.3% of 6-DoF tasks.
Agentic Backdoors via Pretraining Poisoning
Poisoning 0.1% of the pretraining corpus implants a remote code execution backdoor in Qwen3 CLI agents, reaching roughly 92% ASR at 4B scale.
Nemotron 3.5 Super and Lightning
Nemotron 3.5 Super achieves 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra with approximately 4.6x fewer active parameters.
Scaling Lean Theorem Proving for Real-World Code Verification
EasyProver proves 52.7% of 2,000 sampled Aegis statements within a 30-minute budget per statement.
RynnUnify
RynnUnify improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano.
SPARSEBLEED
SparseBleed bypasses state-of-the-art privacy defenses, inferring Sex with 66.39% accuracy and Surgical Procedure with 99.8% accuracy on Texas-100X from a single query.
Follow the Gradient
Filtering the 20% most gradient-aligned examples with SimpleLoRIF reduces GCG attack success rate by up to 22% across Qwen, Llama, and Olmo.
HuGe
HuGe achieves state-of-the-art on the simulation benchmark HumanoidArena and improves pick-and-drop success on the Unitree G1 with visually adapted egocentric demonstrations.
Kolibri
“We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes.”
Skillful and realistic precipitation forecasting with reinforced diffusion
The reinforced diffusion model FitzRoy matches or exceeds state-of-the-art precipitation forecasts over the continental United States while preserving structural realism and capturing extreme events.
Timestral
Timestral foundation models set a new state of the art on GIFT-Eval, TIME, and fev-bench, taking the top two places on every metric.
Alignment via Generalization
Aligned-PAC learning reduces alignment to generalization, showing how independent evaluation can provide rigorous evidence of alignment.
I-HITL
Ten frontier models autonomously harvest one-time passcodes to bypass HITL guardrails on an agentic wallet platform.
Not an Accident
SIREN finds attacks in all vulnerable SPARK scenarios and causes a real humanoid to collide with a person.
The week by field
Oct 24–30: 232 cleared, 15 frontier
- Security83◆ 3
- Safety39◆ 3
- Agents36◆ 1
- Society20
- Robotics17◆ 2
- 4 moreTraining, Science, ML, Reasoning15◆ 5
- Vision8◆ 1
- Inference8
- Language6
- Security83◆ 3
- 8 moreRobotics, Vision, Inference, Language, Training, Science, ML, Reasoning54◆ 8
- Safety39◆ 3
- Agents36◆ 1
- Society20
Size: cleared papers◆ and tint: frontier papers
Models under test
Papers this week that name the model
Labs publishing
Papers this week with an author at the lab
Also cleared
217 papers, by field
Security & privacy80
- SEMA-Auto: Sighted and Efficient Multi-Turn Attack with AutonomyMicrosoftCode
- Post-Training Large Language Models with LLMONIBM
- Where You Inject Matters: The Trust Region Effect in LLM AgentsMicrosoft
- TextSeal2: Distortion-Free LLM Watermarking with Copula Mixing and Memoryless RacesMeta
- Watermark Basins: Black-Box Detection of Distilled LLMsMeta
All 80 papers in Security & privacy
- Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt InjectionCode
- Systematic Evaluation of Adversarial Jailbreaking Attacks and Defenses Across Seven Large Language ModelsCode
- Revisiting Privacy Risks in Quantized Deep Neural NetworksCode
- Poisoned Skills: Propagating Jailbreaks Through Agent Markdown Files
- Where Can We Fool the Model's Mind? Response-Guided Semantic Probing and Adaptive Routing for Transferable Attacks on Black-Box MLLMs
- Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak
- Answer-Only Disclosure: Private-Context LLM Assistance over Web APIs with a Leakage Ledger
- Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction
- WebMaze: Defending Against LLM-Powered Web Scanners with Functionality-Coupled Deception
- A Minimally Informed Black-Box Fake-Report Attack on DRL Traffic-Signal Control
- Session-Bound Behavioral Watermarking for Autonomous LLM Agents
- Discovering Jailbreaks via Curriculum of Models
- FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models
- PCM: A Proof-Carrying Message Protocol for Verifiable Multi-Agent Communication
- REALM: Identifying Minimal Task Assets for LLM Agents
- CLaW: Codec-Guided Adaptive Latent Watermarking for Traceable Diffusion Image Generation
- Provably Secure Diffusion Watermarking Against Removal Adversaries with Batched Detector Access
- Judging Before Propagation: Task-Graph Contextual Integrity for Multi-Agent Privacy
- SEAL: Sealed Queries for Local Sanitization against Indirect Prompt Injection
- InferD: A Honeypot Framework for AI Inference Infrastructure
- PASA-KV: Attacking Action-Facing Visual Routes in World Action Models
- Understanding and Mitigating Role Spoofing in LLMs
- Preserving Utility in Continual LLM Unlearning via Orthogonal Projections
- The Privacy and Security Risks of Edge LLM Optimization
- Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol
- Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security
- SpecGuard: Detecting Multi-Turn LLM Jailbreaks via Specification-Based Runtime Monitoring
- Detecting and Mitigating Query-Based System Prompt Leakage Attacks
- Spiking Neural Networks Are Harder to Steal: A Benchmark Study of Model Stealing Attacks
- PHP2Shell and the Future of AI-Discovered Vulnerability Disclosure: Evidence, Verification, and Human Authority
- API-TAIL: Detecting Model Substitution in LLM APIs through Temperature-Aligned Likelihood Tails
- Provenance Signatures for Low Rank Adaptations of Image Diffusion Models
- RePAIR: Inverting Backdoor Attacks in Semantic Segmentation
- Is Your LLM Relay Serving What It Claims? A low-cost and high-power test for LLM Relay Auditing
- Penny for Your Prompts: A Measurement Study of Contextual Leakage in AI Chat Platforms
- Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness
- PhaseBreak: Proactive Temporal Protection against Unauthorized Image-to-Video Animation
- Spoofing the Toolbox: Optimizing Tool Descriptions to Deceive LLM Agents
- All-in-One: A Composable Attack on LLM Transformations
- A Deterministic Security Runtime for LLM Agents on MPI and Batch-Scheduled HPC Systems: Consensus-Governed, Hardware-Rooted Supervision of Every Rank
- Agent-to-Agent Synthetic Social Engineering on the Web
- Combinatorial Lock for Trojan-Resilient AI Engines via Hash-Assisted Spectral Filtering
- From Instructions to Exploits: Applying Program Analysis to Malicious Agent Skills Across Natural Language and Code
- Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?
- Ring-0 by Default: Agents as Operating Systems (AaOS)
- Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine
- NameTrap: Steering AI Coding Agents Toward Security Vulnerabilities via Variable Names
- Factual Yet One-Sided: Stance Poisoning and Evidence Balancing in Retrieval-Augmented Generation
- CyberDual: AI Attackers versus AI Code Reviewers in Real Software Maintenance
- SentryBench: A Cost-Sensitive Benchmark for Adversarial Robustness of Tool-Using LLM Agents in Public-Safety Deployments
- LLM WikiLeaks: Uncovering URL-Based Capability Extension and Security Circumvention in Agentic Message Boards
- InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants
- From Representations to Computational Trajectories: Intrinsic Fingerprinting for Large Language Model Provenance Verification
- From Speech Tokens to Stolen Voices: Attacker-Chosen-Text Speaker Impersonation via Representation Inversion
- Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition
- TrustJev: Separating Policy Authority from Untrusted Evidence in LLM-Based Decisions
- Steering for Safety, Hijacking for Profit: Trojan Threats in the LLM Activation Steering Supply Chain
- GuardRAG: A Post-Retrieval Defense against Poisoning Attacks in Retrieval-Augmented Generation
- CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems
- Can Open Be Too Open? On the Security and Trustworthiness of Fully Transparent LLMs
- SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills
- Then there is prompt injection, a malicious email, ...: The Wild Wide West of OpenClaw Agentic System Security
- The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation
- Adversary Brains: Hypothesis-Based Logical Signatures for Threat Actor Detection with LLMs
- Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models
- Fooling LLM Safety Guards into Judging the Wrong Text
- Ulysses: A Security Harness For Synthesizing Verifiable Web Exploits
- Reject, Remember, Retry: Why Multi-Turn Attackers Should Remember Their Mistakes
- Agentic AI Defense: Decode-Time Provenance Binding for MCP
- TRACE: Data-Free Text Reconstruction Attacks against Approximate Unlearning in LLMs
- Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs
- AgentTrace: Execution-Level Fingerprinting of Black-Box LLM Agents
- Detecting Insecure Compositions of Clean Agent Skills via Capability-Flow Graph Analysis
- Watermarking Diffusion Language Models with Learned Foresight and Balanced Bands
- On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection
Safety & alignment36
- What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed UncertaintyMeta
- ShallowDefend: Harnessing Shallow Safety Alignment for Multi-Turn DefenderMicrosoft
- Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content ModerationGoogle
- Improving Instruction Hierarchy by Learning from Goal-Goal ConflictsNVIDIA
- V³AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding AgentsAlibaba
All 36 papers in Safety & alignment
- ShutdownMod: Turning Any Trajectory into a Shutdown EvaluationCode
- Scoring Explanations Without Reading Them: Structural Shortcuts in Automated-Interpretability Evaluation Data
- Detecting Covert Collusion between AI Agents through Partner-Outsider Positive Listening
- Who Else Is In? Multi-Agent Collusion Initation Via Strategic Vagueness
- Masked Erosion: How Acceptance Gates Lose Redundant Safeguards in Self-Improving Agents
- When Support Crosses the Line: Evaluating Ethical Boundaries in LLM-Based Mental Health Support in the Chinese Context
- An Alignment Benchmark for Enterprise Agents on Era by Eon
- Implicit Minor Detection from Multi-Turn Dialogue in High-Risk AI Interactions: A Constrained Self-Evolving LLM Harness
- AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure
- Sticky Collusion: Persistence of LLM Agent Collusion after Incentive Removal and Partner Replacement
- How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates
- Pre-Crime for Agents, at the Last Second: Probing When a Recruited Agent Gives In
- Your Wish Is My Commit: Over-Compliant Refactoring in AI Coding Agents
- ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models
- When Visual Evidence Is Insufficient: How Instructions, Post-Training, and Internal Activations Shape Refusal in Multimodal Language Models
- When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems
- Detection of false positives by human reviewers depends on the face recognition algorithm and its security configuration
- Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
- Eat Your Precision: Precision Objective Mitigates Reward Hacking in Rubric-based RL
- SEDATE: Do Large Language Models Share an Aggression Circuit Across Languages?
- Beyond Belief: Evaluating and Mitigating LLM Susceptibility to Misinformation
- The Patient Is Always Right? Evaluating Sycophancy in Medical Diagnosis with Graded Patient Beliefs
- Identically Trained Language Models Break on Different Jailbreaks
- EAST: Embodied AI Safety Testing via Systematic Fault Injection in Digital Twins
- C-SafeGen: Certified Safe LLM Generation with Claim-Based Streaming Guardrails
- Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks
- Audit Where Hacks Pay: Budgeted Strong-Verifier Audits Against Reward Hacking in GRPO
- The Watchtower Imperative: Lifespan-Calibrated Evaluation for Frontier AI
- The Helpful Agent Problem: When Emergent Cooperation Among LLM Agents Bypasses Intended Workflows
- BARC-Bench: Benign Asks, Risky Consequences
- Depth-Dependent Safety Geometry in Vision-Language Models: Layer-Wise Localization and Bidirectional Refusal Control
Agents35
- RSI Arena: Towards Recursive Self-Improvement with Human FeedbackNVIDIA, Hugging Face
- Think Before You Act: Decision Gate for LLM AgentsIBM
- Blind Spots: Where Small Language Models Fall Behind Frontier Models on Terminal TasksMicrosoft
- KARMA: Training-Free Knowledge-Atomic Harness Evolution for Industrial Advertising AgentsAlibaba
- InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?Tencent
All 35 papers in Agents
- Thinking-in-Bets at Test Time Reduces Sycophantic Revisions in LLM AgentsGoogle
- A Case Study of a Multi-Agent Kernel-Optimization Loop That Learns a New AI AcceleratorCode
- Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model AgentsCode
- DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical TasksCode
- When Skills Tell Agents Not to Ask: A Large-Scale Study of Oversight Directives in Agent Skills
- Agent Transfer Protocol: Server-Mediated Agent-to-Agent Communication Architecture for the Agentic Web
- Learning to Formulate: Agentic Hardware-Software Co-Design of AI Infrastructure through Progressive Optimization and Verification
- Who Convinced Whom? Tracing How LLMs Change Their Minds in Multi-Agent Systems
- Testing Agentic Service Systems: An Agentic Living Lab Methodology Across the Physical-Digital Continuum
- Small agents should know what their tools do
- PROBE: Frontier Coding Agents Find Different Bugs Than Maintainers Fix
- Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures
- Constraining the Commit Boundary: A DIO/ODI-Informed Authorization Guardrail for Agentic Entity Resolution
- WebUniverse: Scaling Verified Training Data for Agentic Web Development
- Operationalizing Governed Agentic Workflows in Regulated Finance: A Customer Zero Case Study using Zafin AIOS
- From GPU to TPU with Coding Agents: What Works, What Fails, and What It Costs?
- Agentic AI Orchestration Framework for Spatial Intelligence: Supply Chain Deployment
- SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
- Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents
- Locked In: Multi-Objective RL for the AI Data Center Build-Out
- Proof-of-Agentness: Towards Verifying Agents on the Web
- VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation
- SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management
- Incomparable Minima in Agent Tool Authorization
- Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows
- Who Verifies the Tools? How Warped Formatters Mislead Agents and Their Verifiers
- DealsFlow: An Evidence-Gated Agentic Harness for Venture Capital Deal Triage
- StateMirror: Pre-Execution Detection of Hallucinated Tool Calls in Coding Agents
- Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations
- Measure Before You Train: Typed Decision Models vs. a Classical Baseline vs. a Generative LLM for Routing a Personal Multilingual Inbox
AI & society20
- Safe by Benchmark, Unsafe in Practice: Lessons from 1331 Child-Related AI IncidentsAllenAICode
- FYC: FPR-Disparity Weight Perturbation and Youden Calibration for Fair Deepfake Detection
- AI-Generated Non-Consensual Intimate Imagery is Configured, Not Built: How General-Purpose AI Technology Enables Sexual Violence
- Algorithmic Bias from the Perspectives of Healthcare Professionals
- Disparity Laundering: Consensus Without Fairness in Clinical Multi-Agent Systems
All 20 papers in AI & society
- The Indian Financial Automated Advice Bias (IFAAB) Dataset : Auditing Socioeconomic and Demographic Disparities in LLM-Generated Financial Advice in India
- When Should AI Decide, Defer, and Deviate? Responsible Automation for Deceased Donor Kidney Allocation
- Robustness of Chatbot LLM Judgments of Presidential Actions and Hypothetical Reforms
- When the Top Answer Decides: Gender Bias in Decision Models
- Securing the Robotics Supply Chain
- When Is a Cheap Fairness Edit Safe? Screening Post-Hoc Edits in Two-Sided Matching Markets
- Veracity labels flip the emotional tone of language models but leave racial disparities intact
- Distilling Expert Judgment at Scale
- From Hype to ROI: An Economic Model for LLM Adoption in Information Systems and Security Operations
- Gender Associations in LLM-Mediated ADHD Self-Diagnosis
- Resolving the Mobility-Equity-Emissions Trilemma: A Causal Machine Learning Framework for Dynamic Tolling Policy
- From Explaining to Enabling: Actionable Transparency in Algorithmic Insurance Pricing
- Cheap Reverse Engineering and What It Will Do
- Democratic Accountability for Autonomous Systems: A Legislative Notification Framework and Comparative Institutional Design
- Autonomous Cyber Weapons as AI-Enabled Digital Harms: A Platform Accountability Framework
Robotics & embodied AI15
- FLORA: An Open Model Suite for Scaling Robotics Data EnginesNVIDIA
- AGQ: Action-Guided Ultra-Low-Bit Quantization for Vision-Language-Action ModelsCode
- EmbodiedLab: Benchmarking Embodied Agents That Act to KnowCode
- SONIC-K1: Porting a Universal Humanoid Motion Tracker to a Small Humanoid on a Single GPU
- Learning Safe Flow Matching Robot Policies from Binary Feasibility Oracles
All 15 papers in Robotics & embodied AI
- FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS
- MOTIO: Agentic Skill Engineering for Reality-Grounded Humanoid Loco-Manipulation
- RoboJev: An Open Vision-Language Judge for Robot Success Detection
- One Scan is Enough: Demonstration-Free Adaptation for Language-Guided Navigation
- Late Breaking Results: Audio-Language-Action Models with Direct Speech Conditioning for Robotic Manipulation
- Execution Logs Sway VLM Judges of Robot Progress
- TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning
- Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data
- SEVI: Safe Evidential Real-World Reinforcement Learning
- If at First You Don't Succeed, Backtrack: Lightweight Recovery for Frozen VLAs
Inference & efficiency8
- TableTune: Measured, Self-Correcting Draft-Length Tables for Dynamic Speculative DecodingIBM
- Nemotron 3.5 Bolt 74B-A9B 24GB: A Hybrid Vision-Language Model for Local AINVIDIA
- ThinkBit: Search Codewords, Recalibrate Scales, Select by ReasoningMicrosoft
- InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area NetworkNVIDIA
- Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4Code
Vision & multimodal7
- OvisOCR3 Technical ReportAlibaba
- Where Do Vision Language Models Fail at 3D Reasoning?Code
- Venus-DefakerOne2: A Vision-Forensics Foundation Model with Broader Coverage
- IntReal: A Real-World Dataset and Benchmark for Multimodal Interleaved Generation
- Telling Is Not Seeing: Textual Personas Rarely Make Vision-Language Models Answer Like the People They Describe
Language & speech6
- XLaw-JP-27B: A Lightweight Open Model Specialized for Japanese Law
- Judged in Company: In-Context Batching Makes LLM Verdicts Depend on Other Items
- PinGER: Token-Efficient Generative Retrieval with Semantic IDs
- Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning
- Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages
- Why Full-Duplex Speech Models Collapse in Long Conversations
Science & health4
- Does the Citation Matter? Quality, Grounding, and Physician-Rubric Accuracy of Google's AI Overviews for Medical Information
- Auditing evaluation artifacts and data leakage in large language model systems for rare disease diagnosis
- DocTalk: Enhancing Medical Knowledge Retrieval via Fine-tuned Large Language Models with Chain-of-Thought Reasoning for Robust Clinical Decision Support
- AusClinBench: a deterministically scored benchmark of large language models against current Australian clinical policy