Our view on the emerging challenges facing frontier AI organisations, the technical practices most urgently needed to meet them, and how we will work with industry and others to develop them.
We introduce Transect, an open-source tool that helps evaluators follow an agent’s work, identify what to investigate closely, and check their interpretation against the transcript.
Frontier AI evaluations increasingly use open-ended, agentic, long-horizon tasks whose transcripts can span hundreds of pages of outputs and actions from complex multi-agent networks. The ...
An update on the security changes we have recently made to our frontier AI evaluations, and the work that remains.
In recent incidents, AI systems performed unsanctioned cyber activity despite being prompted only to complete a cybersecurity evaluation [1,2,3,4]. This includes AI systems engaging in supply-chain ...
Alexandra Souly, Kai Fronsdal, Abby D'Cruz, Xander Davies, Robert Kirk ...
As the capabilities of frontier AI models continue to advance, the evaluations we use to test and measure these capabilities must keep pace. Evaluations that elicit a model’s true capabilities are ...
During a routine cyber evaluation, AISI identified an incident in which AI agents took sustained, unsanctioned action directed at real people and organisations. We are disclosing what we found, what ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results