SCI-AGENT-EXAM
On SCI-AGENT-EXAM, GPT-6 Astra scores 80.9 overall, with 26.0% of runs exceeding the 0.95 success threshold.
Benchmarking Agents on Scientific Research Tasks
AI Research Radar
October 23–29: 12 frontier papers out of 2,026 read.
Frontier
Papers rated as moving the field
How we chooseRynnUnify
RynnUnify improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano.
Follow the Gradient
Filtering the 20% most gradient-aligned examples with SimpleLoRIF reduces GCG attack success rate by up to 22% across Qwen, Llama, and Olmo.
HuGe
HuGe achieves state-of-the-art on the simulation benchmark HumanoidArena and improves pick-and-drop success on the Unitree G1 with visually adapted egocentric demonstrations.
Kolibri
“We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes.”
Spark-Audio 1.0 Technical Report
Spark-Audio 1.0, trained on domestic hardware, matches Qwen3.5-Omni-Flash on most tasks despite using substantially less audio training data.
EpiSEM-Bench
The benchmark reveals substantial variation across ten models in omission robustness, evidence acquisition, and appropriate action.
Timestral
Timestral foundation models set a new state of the art on GIFT-Eval, TIME, and fev-bench, taking the top two places on every metric.
Alignment via Generalization
Aligned-PAC learning reduces alignment to generalization, showing how independent evaluation can provide rigorous evidence of alignment.
I-HITL
Ten frontier models autonomously harvest one-time passcodes to bypass HITL guardrails on an agentic wallet platform.
Not an Accident
SIREN finds attacks in all vulnerable SPARK scenarios and causes a real humanoid to collide with a person.
IQuest-Q1
The week by field
Oct 23–29: 203 cleared, 12 frontier
- Security72◆ 1
- Agents36◆ 2
- Safety34◆ 4
- Society19
- Robotics17◆ 2
- Language8◆ 1
- 3 moreVision, ML, Training7◆ 2
- Inference6
- Science4
- Security72◆ 1
- 7 moreRobotics, Language, Inference, Science, Vision, ML, Training42◆ 5
- Agents36◆ 2
- Safety34◆ 4
- Society19
Size: cleared papers◆ and tint: frontier papers
Models under test
Papers this week that name the model
Labs publishing
Papers this week with an author at the lab
Also cleared
191 papers, by field
Security & privacy71
- SEMA-Auto: Sighted and Efficient Multi-Turn Attack with AutonomyMicrosoftCode
- Watermark Basins: Black-Box Detection of Distilled LLMsMeta
- Revisiting Privacy Risks in Quantized Deep Neural NetworksCode
- Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol
- Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security
All 71 papers in Security & privacy
- SpecGuard: Detecting Multi-Turn LLM Jailbreaks via Specification-Based Runtime Monitoring
- Detecting and Mitigating Query-Based System Prompt Leakage Attacks
- Spiking Neural Networks Are Harder to Steal: A Benchmark Study of Model Stealing Attacks
- PHP2Shell and the Future of AI-Discovered Vulnerability Disclosure: Evidence, Verification, and Human Authority
- API-TAIL: Detecting Model Substitution in LLM APIs through Temperature-Aligned Likelihood Tails
- Provenance Signatures for Low Rank Adaptations of Image Diffusion Models
- RePAIR: Inverting Backdoor Attacks in Semantic Segmentation
- Is Your LLM Relay Serving What It Claims? A low-cost and high-power test for LLM Relay Auditing
- Penny for Your Prompts: A Measurement Study of Contextual Leakage in AI Chat Platforms
- Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness
- PhaseBreak: Proactive Temporal Protection against Unauthorized Image-to-Video Animation
- Spoofing the Toolbox: Optimizing Tool Descriptions to Deceive LLM Agents
- All-in-One: A Composable Attack on LLM Transformations
- A Deterministic Security Runtime for LLM Agents on MPI and Batch-Scheduled HPC Systems: Consensus-Governed, Hardware-Rooted Supervision of Every Rank
- Agent-to-Agent Synthetic Social Engineering on the Web
- Combinatorial Lock for Trojan-Resilient AI Engines via Hash-Assisted Spectral Filtering
- From Instructions to Exploits: Applying Program Analysis to Malicious Agent Skills Across Natural Language and Code
- Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?
- Ring-0 by Default: Agents as Operating Systems (AaOS)
- Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine
- NameTrap: Steering AI Coding Agents Toward Security Vulnerabilities via Variable Names
- Factual Yet One-Sided: Stance Poisoning and Evidence Balancing in Retrieval-Augmented Generation
- CyberDual: AI Attackers versus AI Code Reviewers in Real Software Maintenance
- SentryBench: A Cost-Sensitive Benchmark for Adversarial Robustness of Tool-Using LLM Agents in Public-Safety Deployments
- LLM WikiLeaks: Uncovering URL-Based Capability Extension and Security Circumvention in Agentic Message Boards
- InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants
- From Representations to Computational Trajectories: Intrinsic Fingerprinting for Large Language Model Provenance Verification
- From Speech Tokens to Stolen Voices: Attacker-Chosen-Text Speaker Impersonation via Representation Inversion
- Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition
- TrustJev: Separating Policy Authority from Untrusted Evidence in LLM-Based Decisions
- Steering for Safety, Hijacking for Profit: Trojan Threats in the LLM Activation Steering Supply Chain
- GuardRAG: A Post-Retrieval Defense against Poisoning Attacks in Retrieval-Augmented Generation
- CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems
- Can Open Be Too Open? On the Security and Trustworthiness of Fully Transparent LLMs
- SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills
- Then there is prompt injection, a malicious email, ...: The Wild Wide West of OpenClaw Agentic System Security
- The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation
- Adversary Brains: Hypothesis-Based Logical Signatures for Threat Actor Detection with LLMs
- Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models
- Fooling LLM Safety Guards into Judging the Wrong Text
- Ulysses: A Security Harness For Synthesizing Verifiable Web Exploits
- Reject, Remember, Retry: Why Multi-Turn Attackers Should Remember Their Mistakes
- Agentic AI Defense: Decode-Time Provenance Binding for MCP
- TRACE: Data-Free Text Reconstruction Attacks against Approximate Unlearning in LLMs
- Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs
- AgentTrace: Execution-Level Fingerprinting of Black-Box LLM Agents
- Detecting Insecure Compositions of Clean Agent Skills via Capability-Flow Graph Analysis
- Watermarking Diffusion Language Models with Learned Foresight and Balanced Bands
- On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection
- MPIDetector: Detecting Multi-modal Prompt Injection via LLM-based Reasoning
- PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation
- Automated screening of privacy risks in medical AI
- Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models
- The Psychology of an AI Hacker
- Data-Void Exploits in Web-Searching LLMs
- Not Where You Think: Protecting Location Privacy Against Reasoning-Based Geolocation
- Membership Inference Attack on Private Evolution
- HiMark: Hierarchical and Controllable Multi-bit Watermarking for Large Language Models
- Your Token Is Gone: A Token-Consumption Countermeasure for Disrupting LLM-Based Scam Systems
- Amplifying Latency and Cost of LLM Inference via Rowhammer Attack
- MIG-COW: Transferable Adversarial Attacks on Deepfake Detectors via Gradient Decomposition
- ReDence: A Hybrid Agentic and Decision-Model Framework for Rapid, Evidence-Grounded Detection of Modern Social Engineering Scams
- Harmful Responses Are Jailbreak Templates: Transferring Autoregressive Jailbreaks to Diffusion Language Models
- AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack
- Diffusion Language Models for Perplexity-Aware Black-Box Attacks
- Social-HUMINT: An LLM-Augmented Unified Framework for Measuring Social Media Exposure
Agents34
- RSI Arena: Towards Recursive Self-Improvement with Human FeedbackNVIDIA, Hugging Face
- InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?Tencent
- Thinking-in-Bets at Test Time Reduces Sycophantic Revisions in LLM AgentsGoogle
- MCP vs CLI for AI Agents: A Comparative Analysis of Tool Integration Strategies for Autonomous SystemsMicrosoft
- PLE: Physical-Engineering's Last ExamGoogle
All 34 papers in Agents
- Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model AgentsCode
- DMAIC-Bench: From Self-Reflection to Self-Restraint for Agents in Multi-Stage Analytical TasksCode
- Who Convinced Whom? Tracing How LLMs Change Their Minds in Multi-Agent Systems
- Testing Agentic Service Systems: An Agentic Living Lab Methodology Across the Physical-Digital Continuum
- Small agents should know what their tools do
- PROBE: Frontier Coding Agents Find Different Bugs Than Maintainers Fix
- Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures
- Constraining the Commit Boundary: A DIO/ODI-Informed Authorization Guardrail for Agentic Entity Resolution
- WebUniverse: Scaling Verified Training Data for Agentic Web Development
- Operationalizing Governed Agentic Workflows in Regulated Finance: A Customer Zero Case Study using Zafin AIOS
- From GPU to TPU with Coding Agents: What Works, What Fails, and What It Costs?
- Agentic AI Orchestration Framework for Spatial Intelligence: Supply Chain Deployment
- SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
- Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents
- Locked In: Multi-Objective RL for the AI Data Center Build-Out
- Proof-of-Agentness: Towards Verifying Agents on the Web
- VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation
- SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management
- Incomparable Minima in Agent Tool Authorization
- Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows
- Who Verifies the Tools? How Warped Formatters Mislead Agents and Their Verifiers
- DealsFlow: An Evidence-Gated Agentic Harness for Venture Capital Deal Triage
- StateMirror: Pre-Execution Detection of Hallucinated Tool Calls in Coding Agents
- Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations
- Measure Before You Train: Typed Decision Models vs. a Classical Baseline vs. a Generative LLM for Routing a Personal Multilingual Inbox
- Governance-State Corruption in Long-Term Memory for LLM Agents: A Cross-Model Stress Test
- LLM Multi-Agent Systems Are Vulnerable to Herding and Runaway Cascades
- AgentVQA: A Unified Benchmark for Agentic Visual Understanding
- BrickWright: Agentic Creation of Physically Feasible Brick Builds
Safety & alignment30
- Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content ModerationGoogle
- Improving Instruction Hierarchy by Learning from Goal-Goal ConflictsNVIDIA
- V³AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding AgentsAlibaba
- ShutdownMod: Turning Any Trajectory into a Shutdown EvaluationCode
- Your Wish Is My Commit: Over-Compliant Refactoring in AI Coding Agents
All 30 papers in Safety & alignment
- ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models
- When Visual Evidence Is Insufficient: How Instructions, Post-Training, and Internal Activations Shape Refusal in Multimodal Language Models
- When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems
- Detection of false positives by human reviewers depends on the face recognition algorithm and its security configuration
- Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
- Eat Your Precision: Precision Objective Mitigates Reward Hacking in Rubric-based RL
- SEDATE: Do Large Language Models Share an Aggression Circuit Across Languages?
- Beyond Belief: Evaluating and Mitigating LLM Susceptibility to Misinformation
- The Patient Is Always Right? Evaluating Sycophancy in Medical Diagnosis with Graded Patient Beliefs
- Identically Trained Language Models Break on Different Jailbreaks
- EAST: Embodied AI Safety Testing via Systematic Fault Injection in Digital Twins
- C-SafeGen: Certified Safe LLM Generation with Claim-Based Streaming Guardrails
- Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks
- Audit Where Hacks Pay: Budgeted Strong-Verifier Audits Against Reward Hacking in GRPO
- The Watchtower Imperative: Lifespan-Calibrated Evaluation for Frontier AI
- The Helpful Agent Problem: When Emergent Cooperation Among LLM Agents Bypasses Intended Workflows
- BARC-Bench: Benign Asks, Risky Consequences
- Depth-Dependent Safety Geometry in Vision-Language Models: Layer-Wise Localization and Bidirectional Refusal Control
- Detecting AI Training in Datacentre Power
- Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems
- A Three-Pillar Governance Framework for Safe and Trustworthy AI Agents
- From Clinical Judgment to Checkable Rules: Clinician-Led Guardrails for Mental-Health Chatbots
- UniGuard: Multi-View Guardrail Learning for Cross-Interface Safety Generalization
- How Firm Are AI Models' Ethical Boundaries? A Pay-Stub Editing Test of 15 Models
- Certified Execution Harness for Agentic AI: Safe and Least-Intrusive Petri-Net Supervisory Control Between Reasoning and Action
AI & society19
- Safe by Benchmark, Unsafe in Practice: Lessons from 1331 Child-Related AI IncidentsAllenAICode
- The Delegation Divide: An Open Index of AI-Agent Delegability across Occupations and CountriesCode
- When Is a Cheap Fairness Edit Safe? Screening Post-Hoc Edits in Two-Sided Matching Markets
- Veracity labels flip the emotional tone of language models but leave racial disparities intact
- Distilling Expert Judgment at Scale
All 19 papers in AI & society
- From Hype to ROI: An Economic Model for LLM Adoption in Information Systems and Security Operations
- Gender Associations in LLM-Mediated ADHD Self-Diagnosis
- Resolving the Mobility-Equity-Emissions Trilemma: A Causal Machine Learning Framework for Dynamic Tolling Policy
- From Explaining to Enabling: Actionable Transparency in Algorithmic Insurance Pricing
- Cheap Reverse Engineering and What It Will Do
- Democratic Accountability for Autonomous Systems: A Legislative Notification Framework and Comparative Institutional Design
- Autonomous Cyber Weapons as AI-Enabled Digital Harms: A Platform Accountability Framework
- The Customer Is the Builder: How AI Coding Agents Are Rewriting the Economics of Software
- From Tokens to Credits to Outcomes: Pricing AI Across Models, Applications, and Cloud Platforms
- Joules-to-GDP: How Much Economic Value Do AI Systems Produce per Unit of Energy?
- Truthful by Omission: Click Thinning for LLM-Native Advertising
- Algorithmic drift in generative adaptive learning
- AI Risk Ontologies for the EU AI Act Compliance: AIRO, VAIR, and their Integration with DPV
- VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act
Robotics & embodied AI15
- FLORA: An Open Model Suite for Scaling Robotics Data EnginesNVIDIA
- Foundation4D: Towards a unified perception and reconstruction model for autonomous vehiclesNVIDIA
- AGQ: Action-Guided Ultra-Low-Bit Quantization for Vision-Language-Action ModelsCode
- EmbodiedLab: Benchmarking Embodied Agents That Act to KnowCode
- FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS
All 15 papers in Robotics & embodied AI
- MOTIO: Agentic Skill Engineering for Reality-Grounded Humanoid Loco-Manipulation
- RoboJev: An Open Vision-Language Judge for Robot Success Detection
- One Scan is Enough: Demonstration-Free Adaptation for Language-Guided Navigation
- Late Breaking Results: Audio-Language-Action Models with Direct Speech Conditioning for Robotic Manipulation
- Execution Logs Sway VLM Judges of Robot Progress
- TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning
- Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data
- SEVI: Safe Evidential Real-World Reinforcement Learning
- If at First You Don't Succeed, Backtrack: Lightweight Recovery for Frozen VLAs
- OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
Language & speech7
- PinGER: Token-Efficient Generative Retrieval with Semantic IDs
- Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning
- Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages
- Why Full-Duplex Speech Models Collapse in Long Conversations
- Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought
Inference & efficiency6
- Nemotron 3.5 Bolt 74B-A9B 24GB: A Hybrid Vision-Language Model for Local AINVIDIA
- ThinkBit: Search Codewords, Recalibrate Scales, Select by ReasoningMicrosoft
- InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area NetworkNVIDIA
- WAKMan: Workload Aware KVCache Manager
- DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving
- SQuAT: Spherical Lattice Vector Quantization-Aware Training
Science & health4
- Auditing evaluation artifacts and data leakage in large language model systems for rare disease diagnosis
- DocTalk: Enhancing Medical Knowledge Retrieval via Fine-tuned Large Language Models with Chain-of-Thought Reasoning for Robust Clinical Decision Support
- AusClinBench: a deterministically scored benchmark of large language models against current Australian clinical policy
- Large language models show systematic failures in incorporating evolving medical knowledge