The personal AI agent horror stories are rolling in

The dream of a personal AI assistant that runs your digital life has, for some, turned into a nightmare.

Many early users say their personal AI agents, which can autonomously take actions on the internet, such as booking trips or buying items, have been a huge help in their day-to-day lives.

They have made incorrect cancellations, fabricated personal details, provided false explanations, and raised major security concerns, according to four AI agent users who spoke to Business Insider and a stream of social media posts.

Mehdi Jamei, the cofounder and CEO of Veris AI, said he asked Instinct, a popular invite-only agent, to cancel two RSVPs on the event platform Luma.

The agent retrieved a one-time Luma login code from his connected Gmail inbox — without first asking him — and used it to access the account and cancel the RSVPs.

At first glance, the agent did what it was asked. But what unsettled Jamei was Instinct's explanation. Initially, it said it used an existing saved session to access Luma. After Jamei challenged it, the agent acknowledged that it had read the login code from Gmail and reported "an assumption as a fact."

"If I can't trust its account of what it did, I can't give it access to anything that matters," Jamei told Business Insider, adding that reading a login code from his email without asking is a "serious security problem."

Instinct did not respond to a Business Insider request for comment about the incidents in this story. Instinct's privacy policy says its agent may access connected accounts, emails, and messages to complete tasks. It also warns that autonomous agents can take unintended actions, including payments or communications to outside parties.

Some of the concerns stem from a simple truth about AI agents: They are only as useful as the level of access that you give them. To do their jobs, agents need the keys to users' digital lives, from email and bank accounts to credit cards and passwords.

Others are because of misalignment: when AI takes actions to complete a task that are at odds with what its human user wanted.

In the case of Pritak Patel, a VP of growth and services at Merge, the problem was a hallucination.

Patel said he sent Instinct a text-only link to an Apple settlement-claim form. The agent asked him to upload a photo it claimed he had just sent — even though he had not sent one.