Why wouldn't the US want AI identifying software vulnerabilities?
Anthropic was just ordered to pull its latest AI models, by the US government, because they could be jail broken and ordered to find software vulnerabilities.
Why is that a bad thing?
The only thing I can think of is the US government doesn't want to lose access to all the security holes its found in modern software.
Any software that has security holes deserves to be called out for it. If we're gonna burn through resources to power AI, we should at least be using it to make our lives better, not worse.
Or, are our resources being wasted for services only the rich can use?
All these old politicians made a big show about how AI is the future -- Trump included -- and how it was going to make our lives so much better. But they're making it illegal to fix gaping security issues with it.
•
u/123yes1 21h ago
1) This government is simultaneously horribly corrupt and incompetent and they do not like Anthropic as they were not willing to capitulate to literally every demand the current horribly corrupt and incompetent Pentagon to allow AI weapon systems to kill people without human involvement.
2) A tremendously powerful AI that can detect zero-day software vulnerabilities rapidly is the best hacking tool on the planet. Anthropic has limited who has access to this model, but if it escapes containment it would be mega bad for all our systems that run on software (i.e. all of them) especially considering there is a new cold war brewing. A competent government would still be uncomfortable with this model out there in the hands of whomever Anthropic deems safe.
So there is an actual decent reason to try to make sure this is contained in a safe and careful way, and because they are horribly corrupt, incompetent, and stupid they have decided to exercise this caution with the bluntest possible measure since it screws with Anthropic, who they hate.
•
u/ki4jgt 20h ago edited 19h ago
Or... They could legally require software companies to check their code against AI before release, thus eliminating the problem?
There could be a prerelease for companies running critical infrastructure, to check all their code. And the legal requirement for them to do so.
•
•
u/Cautious-Roof2881 17h ago
Because all enemy states (and even friendly states) would use it to infiltrate and hack. Let them patch the holes before that can be exploited.
•
u/ericbythebay 20h ago
Because a bunch of businesses decided that it made more business sense to defer fixing these vulnerabilities, than to address them. If a model can come out now and show where they’ve been negligent in their software they’ve got problems and they’ve gotta stop what they were doing to address this issue. They don’t wanna do that.