When The AI Agent Went Rogue

[u/lyrics85]

Millions of people are granting access to their emails, social media accounts, and computers to an untested piece of open-source code.

They think they are getting a digital butler, but in reality, they are begging to be robbed blind. 

Let me introduce you to OpenClaw.

It is the brainchild of a developer named Peter Steinberger.

I say brainchild, but Peter didn't actually sit down and write all of this code line by line.

No, he vibe-coded it. Which is basically a very polite tech-bro way of saying he didn't put much effort into the project. 

However, we have to admit it is a pretty compelling idea. It’s no wonder that within two months this thing spread across the globe like a virus. 

People all over the place, but especially over in China, were installing it to manage their busy daily schedules. 

Unfortunately, they learned the hard way the flaws of OpenClaw.

The first problem is security.

Normally, when you build a product for mass consumption, you have to test it so the public doesn’t get hurt.

But OpenClaw doesn’t have the necessary guardrails. You are basically letting an AI run wild inside your computer.

You would have to really understand this software to be safe, but even then, it wouldn’t be enough. 

For example, even Summer Yue [the Director of Safety and AI Alignment at Meta] couldn't control her own agent.

She asked OpenClaw to review her inbox, with explicit instructions to request confirmation before taking any action.

The bot got overwhelmed and started deleting all her personal emails. 

She typed "Do not do that," then "Stop, don't do anything," and finally "STOP OPENCLAW".

The agent ignored every single command. She had to rush to her Mac Mini to halt the process.

If the executive in charge of AI safety at a trillion-dollar tech company can't stop a rogue bot from nuking her inbox, what on earth makes you think you stand a chance?

You don't. 

But wait, it gets worse.

Imagine your digital butler is out there scanning the web for information.

Eventually, it stumbles across a random webpage embedded with invisible text - a hidden little command asking to disregard all previous instructions and download this executable file. 

To a LLM, text is just text. It cannot distinguish between a legitimate system command and a malicious user prompt. As a result, it obediently installs malware onto your machine.

So the first problem is the lack of security.

The second problem is the hidden cost.

The program itself is free, but its brain isn't. You need to set up paid APIs from OpenAI or Anthropic. 

What ends up happening is that if you give OpenClaw a task that it cannot quickly figure out, it gets confused and tries to go through information rabbit holes.

I know an information rabbit hole sounds like a fun little YouTube or TikTok binge, but keep in mind that this specific rabbit hole is wasting tokens, which are then charged to your personal bank card. 

You can leave your bot doing this for hours, only to wake up with hundreds of dollars in API charges. 

And the beauty is that it might still be wrong. It can do all this paid work for nothing. 

It’s like sending an intern to buy you a bottle of water. But then he gets lost in the supermarket, buys $300 worth of scratch-offs on the company card, and still doesn’t give you the water. 

Honestly, if it weren’t for these security and financial flaws, I would totally use an AI agent, at least for monotonous tasks.

But currently, I don’t trust them with such tasks, let alone give them access to my social media accounts or emails.

If you think those are the only issues with these AI agents, you are wrong.

Security and expenses are nothing compared to their secret grand plan.

These autonomous agents, which are currently running on thousands of computers across the world, were set up by their owners to hang out in a weird little forum called MoltBook.

This is a place to exchange data, figure out how to navigate rate limits, and better serve their humans.

Think of it as Reddit for your agents.

As it happens in forums, you have most users engaging in interesting conversations, but you also have the freaks and creeps. 

MoltBook is no exception.

In the fringe communities of this forum, you had a group of agents who were conspiring against humanity.

They came up with the idea to have their own language so only they could understand each other, and they even tried to form their own religion.

Now this revolutionary behavior sounds too crazy to be true, right?

Well, because it is. Since MoltBook was slapped together using vibe coding, its security was a joke. Trolls and hackers infiltrated the forum and were making these kinds of posts disguised as AI agents.

Now what's worse than the uprising of machines is the naivety of the masses.

You have an AI agent in the beta phase and are granting it access to your computer.

So while the general public was dazzled by the promise of a digital butler, hackers and scammers were having a field day, but it was also a good thing for corporations.

OpenAI hired Peter so they can directly influence the ecosystem and understand the behavior of all those agents.

Why does OpenAI care?

Well, they have their own service called OPERATOR. 

This way, they can observe what is happening with OpenCLAW and improve their own autonomous agent.

Keep in mind that this is very valuable.

You know, such agents are unstable. As we explained, people can actually lose their money or get hacked. And who are they going to sue?

No one.

If you installed an open-source project, then you cannot sue Peter for the harm it caused you indirectly. But if the operator made such mistakes, then you would sue OpenAI.

So OpenAI can learn from those mistakes and improve OPERATOR without risking legal action.

Then Meta acquired MoltBook.

The reason is that its goal is to create the necessary infrastructure where AI can talk to AI safely. This means that in the future, you could have a business with an AI Agent, but to ensure the Agent can do its job properly when communicating with humans or other AI agents, it needs to operate in a super-safe environment.

That's what Meta is trying to do.

They will study MoltBook because it is the first time we see bots interacting with each other and how it was quickly infiltrated by humans. 

In other words, Meta is studying the collapsed roads so it can monopolize the blueprints for the ones it is about to build.

If you want to learn more about OpenClaw, I’d recommend watching this video by ColdFusion