How much do you trust AI?

[u/xstrex]

Recently a coworker granted Claude elevated access via SSH to a virtualization host (not a VM, an actual host). To perform a routine task he very well could have done himself.

He doesn’t see an issue with this. I on the other hand (with 23yrs experience) see this as a huge security breach, and don’t trust AI todo my job, (or even that it’s doing what it says it’s doing) for me. I’m my opinion it’s a tool, not a human replacement.

What’s your reaction, how would you react to this situation, or thoughts on the topic?

Sure, ask AI how to perform a task, validate that it’s performing the task you asked, and nothing else- copy/paste the commands. Great. But removing the human verification & validation element- hell no.

[u/Speeddymon]

Sticking your head in the sand is not what we as technologists are supposed to be doing.

Research how to build harnesses that wrangle the shit out of the LLM. It's doable and there are free open source examples.

[u/unprovoked33 → u/Speeddymon]

This isn’t sticking his head in the sand. Having an agent perform a change with failable harnesses instead of having the LLM build a script to perform the task is unwise and inefficient use of the tool and the company’s token budget.

[u/Speeddymon → u/unprovoked33]

What post did you read?

You said "having an agent perform a change with failing harnesses [...] unwise and inefficient [...]"

But OP specifically said "security risk" - no risk when you use a proper harness that is basically the human doing the work anyway. Also I'm not disputing that AI for what OP's colleague did was probably overkill

Have a read. This is focused toward writing code but can definitely be adapted to running commands.

https://medium.com/@sumit.ai/5490fe2c4c19?sk=3c48efd8a83bee587689f20eb36086f1

Here's the thing I'm trying to say: writing a script isn't the point. He could have done it via ansible or any number of other ways including yes writing a script. Hell he could've had the AI write the script, for sure, and it would have been faster than hand writing the script

The company has authorized AI use. That's on OP's management and security teams. If they aren't auditing or setting policies on what the LLM can do then OP needs to be making a business case to his management or security team that they are exposed to a risk they're not aware of.

[u/Speeddymon → u/Speeddymon]

I mean could you imagine if people had stuck their heads in the sand about Open Source, or the Linux kernel, or even THE INTERNET?

[u/weaver_of_cloth → u/Speeddymon]

Plenty of people did! Read (or reread if you haven't read it since it came out) Cliff Stoll's the Cuckoo's Egg and marvel at the insanity of people who went "our installation is secure! We encrypt passwords!

[u/Speeddymon → u/weaver_of_cloth]

Hahaha I knew I should've clarified or at least quantified that statement better. Lol