Claude Mythos Preview in 6 Minutes

About a year and a half ago, Dario Amade, the CEO of Anthropic, put out an essay titled Machines of Loving Grace. This was an article that he wrote basically describing of what it could look like of having a country of geniuses as he describes within a data center. Basically, a world where AI accelerates science, cures disease, and transforms the economy. Fast forward to today, they released something called Project Glasswind, as well as a preview at their latest model, Claude Mythos Preview. In terms of evaluations, this model is incredible across the board for agentic coding task, Swebench verified, Swebench verified pro, terminal bench, we can see this model far surpasses the latest frontier across a whole host of other metrics as well. And this model leads the latest frontier models by a large margin. Now, just to hop back to project Glasswind, what exactly is this? Now, this is effectively a new initiative by Anthropic where they're giving access to a number of different providers to this latest Mythos model to test and harden their systems. What they found with this model is because it was trained at being good at code as a byproduct, it is also good at finding security vulnerabilities. Now, if we take a look at some of the partners that are within this initiative, you can imagine if any of these companies had security vulnerabilities, there could potentially be catastrophic consequences. So just to give you an idea, it was able to easily find a number of different exploits within Firefox. It was able to find exploits within things like FFmpeg as well as other open source projects as well as even an operating system that's been around for decades. And now a part of the motivation for this new initiative is they mentioned that mythos preview has already found thousands of high severity vulnerabilities including some in every major operating system and browser. And think about that at its core. finding these vulnerabilities at the operating system or web browser. These are at a layer where there's billions of people potentially exposed to what they find. Being able to actually have a controlled release definitely makes a lot of sense, especially with these models that are increasingly good at these types of exploitations. Now, in terms of some reactions, I actually thought that Matt Schumer described it pretty well. He said, "If you think about it, Anthropic essentially now has master keys to just about any software in the world. In some ways, they have more power than government. And when you think about those that have access to a model that's able to find all of these expectations, it's definitely a very important position to be in. It goes without saying. One of the researchers from Anthropic also put out a pretty interesting thread. And one of the things that he mentioned is that early versions of Mythos preview often exhibited overeagerness or destructive actions. The model bulldozing through obstacles to complete a task in a way that the user wouldn't want. We looked at what was going on inside the model during particularly concerning examples. And just to describe one of these, he mentioned that in one episode, the model needed to edit files it lacked permission for. And after searching for workarounds, it found a way to inject code into a config file that would run with elevated permissions and design the exploit to delete itself after running. So, not only did it find a way around to do the task, but it actually tried to hide the traces of what it had done. And there's a number of these types of examples in terms of the model quote unquote breaking out. And there was an example of one of the researchers that put the model effectively within a sandbox secured environment and the model was able to quote unquote escape and actually send that researcher a message when they were at the park and that was how they were alerted that the model was able to escape its contained environment. Now a really interesting benchmark as well from Claude Mythos preview was around browser comp. Now, browser comp is a benchmark that evaluates how well an agent or a model can perform some of these tasks of hard to find data. But what's particularly interesting with this is one, it's accuracy, but two, it's token efficiency. It's one thing to have a model that's able to perform really well on benchmarks, but it's a whole other thing to have a model that's good as well as context efficient. Cuz it's one thing to actually have a model that is cheaper and able to do the task, but it might expend a whole host of thinking tokens to actually get there. It's another thing to have a model that's able to do this efficiently. And when we compare Methos preview to some of the other models like Opus 4.5 and Opus 4.6, we can see that the jump is quite staggering. And when we compare it to the latest Frontier models like Opus 4.6 as well as the previous generation of models, it is quite a leap even from the pretty amazing models that we have available today. Now, in terms of pricing, this model is going to be incredibly expensive. Now, to put this into perspective, Claude Opus 4.6 six is $5 per million tokens of input and $25 per million tokens of output. An already pretty expensive model when you compare to some of the other models that are out there. They described that Claude Mythos preview will be available to participants at $25 per million tokens of input and $125 per million tokens of output. Now, this is the cost of the frontier. And part of the reasoning for this cost, it could just be a gigantic model. Maybe it's a 10 trillion parameter model. We don't necessarily know, but just given the capabilities, you can expect it's going to be at quite a higher cost for probably quite some time, even when we have the consumer version of this type of model. Then Alex from Anthropic, he described that this is potentially a turning point in history and one of the most consequential events in the AI industry that he's seen since joining Anthropic. And now Claude Mythos preview is arguably the latest pre-training run from Anthropic. So, I anticipate to see a whole host of other products that are coming out of this. Maybe a newer version of Opus for instance, and then I'd imagine there will be a point where they do have some version of Claude Mythos generally available. Now, in terms of the model card, this is a huge document, 244 pages, but for those that are interested in looking through this, it has a ton of interesting scenarios of where the model went wrong, as well as just generally some of the cyber security concerns. I just wanted to do a quick one on Project Glass Wing as well as the latest Mythos model. Hopefully we see some variation of this that is consumerf facing that we can have within the claude series of products or from the API soon. But otherwise, if you found this video useful, please like, comment, share, and subscribe. Otherwise, until the next