Anthropic’s latest and frontier model, Claude Mythos Preview, was recently analyzed for its cyber capabilities by the UK AI Security Institute (AISI). The results showed advanced cyberattack capabilities in controlled simulations, marking a big leap over previous frontier models.
The Autonomous Cyberattack Capabilities
In the evaluation, Mythos Preview succeeds 73% of the time on expert-level tasks in the CTF challenge, where AI models should detect and exploit vulnerabilities in target systems. In another, The Last Ones (TLO), of a 32-step process, the AI model autonomously completed a full intrusion from reconnaissance to full takeover that requires 20 hours for humans to accomplish.
However, Claude Mythos Preview completed it in 3 out of 10 attempts, being the first model to solve the TLO, and 22 of 32 steps, surpassing other models.
Limitations and Defensive Gaps in Real-World Enterprise
The results are based on tasks that measure how AI systems can identify and exploit vulnerabilities in digital systems; however, Mythos Preview achieved success in a cyber range, as emphasized by AISI. During testing, it was found that the AI model lacks features in enterprise systems, such as active defenders, defensive tooling, and penalties for triggering security alerts.
Alongside these limitations, the model is least capable of small, weakly defended enterprise systems once initial access is gained. In the future, it will involve evaluating capabilities using ranges simulating hardened and defended, with active monitoring, endpoint detection, and real-time incident response, AISI confirmed.
Anthropic has not released the Mythos Preview model yet; instead, it offers it through a program called Project Glasswing that gives technology, cybersecurity experts, and others access to it.
In all, the testing highlights the growing AI-driven cyber capabilities, while they possess security risks, alongside being a solid improvement in defense. Researchers say that performance boosts the computer and longer runtime budgets, suggesting the capabilities could be improved further under deployment conditions.
What Next?
Finally, AISI suggests starting to follow best security best security practices, such as implementing access controls, logging, and keeping apps up to date. Onwards, it is necessary to be cyber defense ready as these models develop and evolve!
To learn more about the latest updates around the cybersecurity ecosystem, head over to our website now!
Also Read:
Is AI Putting Privacy at Stake? Vibe-hacking Attack on Anthropic’s Claude
