Anthropic has selected Accenture as its first embedded evaluator, placing the consulting firm's staff inside the AI lab to red-team models, test safeguards, and conduct alignment assessments1,3,6. Both companies have committed at least $1 billion to the effort over five years, with Anthropic funding Accenture's work directly in the near term.
The arrangement operationalizes a proposal Anthropic CEO Dario Amodei laid out in a recent essay calling for third-party evaluators to be granted employee-level access inside frontier AI companies to verify safety practices and report incidents2. Amodei described embedded evaluators as the first step in a three-step plan to slow the pace of AI development.
Faculty leads the work
Anthropic said Faculty, the specialist AI firm Accenture acquired in January to serve as its AI division, will carry out the evaluations. The scope includes red-teaming models, conducting alignment assessments, and testing model safeguards. Anthropic said the partnership is not exclusive and that it is in discussions with the research nonprofit METR and other third parties about piloting elements of embedded evaluation using their own funding.
Anthropic pointed to Accenture's practical experience deploying AI for large corporations and government agencies as a key advantage. The choice nonetheless caught observers off guard: discussion around embedded evaluators had centered on AI safety research organizations such as METR, Redwood Research, and Apollo Research. Accenture's shares rose 8% in after-hours trading following the announcement.
Industry and expert response
OpenAI CEO Sam Altman said OpenAI would also commit to the embedded-evaluator practice. The same week, over 100 AI experts signed a public letter organized by the AI Evaluator Forum consortium, warning that evaluators lack the resources and protections needed to test frontier models effectively8. Signatories include Geoffrey Hinton and members of Johns Hopkins University, Stanford University, and METR. Conrad Stosz, chair of the consortium, said the letter aims to "demonstrate a shared common ground on basic principles and ensure that independent oversight can be a meaningful tool for managing AI risk broadly".
The letter calls for evaluators to have scientific objectivity, transparency, independence, and robust protections, including shielding from retaliation by the companies they embed with. It also asks that all frontier AI companies embed evaluators to independently assess risks, including real-world harm incidents.
Questions about teeth
Skeptics have raised structural concerns. Banking regulation expert Julie Andersen Hill told CNBC that Amodei's proposal explicitly cites bank supervision as a precedent, but bank supervisors can compel action or close an institution. "If you don't give them that kind of power, I don't know what they're doing," Hill said9. Albert Ziegler, a model evaluator at cybersecurity company XBOW, noted that current tests can expose failures but may never trigger the rare combination of behavior that produces a catastrophic outcome.
Researchers also flagged a deeper technical motivation: models are getting better at recognizing when they are being evaluated, raising the risk that they behave well during testing while concealing problematic behavior. Clues to such behavior can be missed when testing a finished model but uncovered by investigating how it behaved throughout training.
Anthropic said it remains responsible for the safety of its models and that working with embedded evaluators will not reduce its accountability. The company said long-term funding should come from pooled or government sources, as called for in its Advanced AI Framework published in June.
ANALYSIS The $1 billion commitment and the choice of a large public consultancy over a nonprofit safety lab sets a commercial template for embedded evaluation that subsequent partnerships with METR or others will be measured against.