A plateau is unlikely, at least for cybersecurity. RL scales well here and is replicable outside of Anthropic (rewards are verifiable, so setting up the training environment doesn't require that much cleverness).
The post also points out that the model wasn't trained specifically on cybersecurity, and that it was just a side-effect – so I think there's still a lot of headroom.
It's scary, but there's also some room for cautious non-pessimism. More people than ever can cause billions of dollars of damage in attacks now [1], but the same tools can be used for defensive use. For that reason, I'm more optimistic about mitigations in security vs. other risk areas like biosecurity.
On a topic like cybersecurity, we never win by not looking: One needs top of the line knowledge of how to break a system to be able to protect it. We have that dilemma dealing with human experts: The same government sponsored unit that tells you that you need to update your encryption can hold on to the information and use it to exploit it at their leisure.
Given that it's absolutely impossible to stop people not aligned with us (for any definition of us) from doing AI research, the most reasonable way forward is to dedicate compute resources to the frontier, and to automatically send reasonable disclosures to major projects. It could in itself be a pretty reasonable product. Just like you pay for dubious security scans and publish that you are making them, an LLM company could offer actually expensive security reviews with a preview model, and charge accordingly.
Terrible take. You don't get to push the extinction button just because you think China will beat you to the punch.
>This is the very nature of being a human being. We summit mountains, regardless of the danger or challenge.
No, just no... We barely survived the Cold War, at times because of pure luck. AI is at least as dangerous as that, if not more. We have far exceeded our wisdom relative to our capabilities. As you have so cleanly demonstrated.
You assume there is the option of not pushing the extinction button. Nobody asked chimps if they wanted humans around. This processes are outside control.
I predict they will release it as soon as Opus 4.6 is no longer in the lead. They can't afford to fall behind. And they won't be able to make a model that is intelligent in every way except cybersecurity, because that would decrease general coding and SWE ability
None of them, but prefer ones written with engineering rigor and security in mind. Having an unvetted plugin ecosystem with code that runs unsandboxed is laughably naive
"now at OpenAI" were my original words - they did the equivalent of an acqui-hire and "protected" OpenClaw in a foundation.
In the context of the seemingly aggressive machinations of Anthropic your hair-splitting without clarifying beyond "OpenAI didn't buy OpenClaw" seems itself misleading and rather counter to helping conversations progress.
reply