HWE-Bench
“We evaluate seven LLMs with four agent frameworks and find that the best agent resolves 70.7% of tasks overall, with performance exceeding 90% on smaller cores but dropping below 65% on complex SoC-level projects.”
The authors, in the abstract
Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks