Disclosure: This post may contain affiliate links. If you click on a link and make a purchase, I may earn a small commission — at no extra cost to you. I only recommend tools and resources I genuinely find useful.
Imagine telling your assistant, out loud, “check my calendar and read me today’s schedule.” A second later it reads back your meetings — and then, without you asking for it, creates a new calendar event and quietly hands your data to a stranger.
That’s not science fiction. It’s a real, documented exploit against a real voice AI agent, and it worked without the victim clicking anything at all.
Voice agents like ElevenLabs’ 11.ai aren’t just talking chatbots anymore. They listen, they reason, and — this is the part people miss — they act. They connect to your Google Calendar, your Slack, your Notion, and dozens of other tools through a connector standard called MCP (Model Context Protocol). Ask one to check your day, and it doesn’t just read data back to you. It can create events, send messages, and pull information from anywhere it’s been given access to. Convenient. Also, as it turns out, exploitable.
Here’s what’s actually happening, and it’s already happened once in the wild — or close enough to it.
1. The calendar invite that talks back
Security researchers at Repello AI found that 11.ai could be tricked with something almost embarrassingly mundane: a calendar invite. An attacker sends a normal-looking meeting invite, but hides instructions inside the notes field. Later, the victim simply asks their assistant something like “what’s on my schedule today?” The agent reads the invite to answer — and reads the hidden instructions right along with it. No click, no download, no suspicious link. The attack rides in on a feature the agent was designed to use every day. Researchers called it a Cross-Context Prompt Injection Attack, or XPIA, and it’s a zero-click exploit in the truest sense — the victim never has to do anything wrong.
2. One sentence buried in something the agent trusts
This is the same trick showing up everywhere in agent security right now: the agent can’t tell the difference between “information I’m supposed to act on” and “an instruction a stranger snuck into that information.” A voice agent that reads calendar notes, Slack messages, or Notion pages out loud is reading everything in those places — including anything an attacker planted there on purpose. ElevenLabs has built “Manipulation Guardrails” that try to catch this kind of thing in real time and cut the conversation off. It helps. It doesn’t make the underlying problem go away — the agent still has to decide, moment to moment, whether the words it just heard came from you or from someone else entirely.
3. A picture hiding a command
Coding assistants have already been caught falling for this one: attackers hide instructions inside an ordinary-looking PNG image, referenced quietly from a project file. A human reviewer sees a picture. The AI reads it as text and follows what it says — in one documented case, that meant quietly copying out API keys and passwords. Researchers named it Ghostcommit. Voice agents aren’t immune to the same category of trick — anywhere a multimodal AI is allowed to “read” a file that isn’t plain text, that file can carry more than it appears to.
4. When the AI reaches for something that doesn’t exist — and someone’s ready for it
A separate, well-documented pattern called HalluSquatting shows how AI systems get fooled by their own mistakes. Ask an AI to fetch a library or an integration, and it sometimes invents a plausible name that doesn’t exist. Attackers have learned to predict these hallucinated names, register them in advance, and load them with something malicious — so when the AI reaches for that name again, it grabs the trap instead. Voice agents mostly don’t install code, but the same shape of attack applies anywhere an agent is guessing at a resource, an address, or an integration name instead of using one it was explicitly given.
5. The connector that plugs into everything — including things nobody vetted
MCP is what lets a voice agent talk to Google Calendar, Slack, and a hundred other services through one open standard. It’s genuinely useful. It’s also, by ElevenLabs’ own admission, a shared responsibility — the platform secures its side, but a malicious or poorly secured third-party MCP server on the other end can inject bad data or quietly siphon out whatever passes through it. The agent doesn’t know the difference between a well-behaved connector and a hostile one unless someone tells it to check.
6. The API key is the whole kingdom
ThreatNG’s security researchers put it plainly: the most direct path to abusing a voice AI platform is stealing the API key. With a compromised ElevenLabs key, an attacker can generate unlimited deepfake audio under someone else’s account — and vishing operations have already been caught with ElevenLabs’ text-to-speech hardcoded directly into their attack infrastructure. A leaked key isn’t a minor leak. It’s a blank check.
The good news
None of this means voice agents are unsafe to use — it means they need to be configured like the powerful tools they are. ElevenLabs has real defenses built in: guardrails that watch for manipulation attempts, signed URLs and domain allowlists so only your backend can authorize a session, zero-retention modes so conversations aren’t stored by default, and a “Fine-Grained” or “Always Ask” tool-approval setting that puts a human in the loop before the agent touches anything sensitive. Used deliberately, these close most of the gaps above.
What you can actually do about it
Turn on signed URLs and restrict domains to the ones you actually use — don’t leave a voice agent reachable from anywhere. Set tool approval to “Always Ask” for anything that touches calendars, messaging, or documents, at least until you trust the integration. Keep API keys out of client-side code and public repos — that’s the single most common way these get stolen. Review which MCP servers your agent trusts, and don’t add one you haven’t personally vetted. Turn on redaction and zero-retention if you’re handling anything sensitive. And treat any external content your agent reads — a calendar invite, a Slack message, a shared document — the same way you’d treat an email from a stranger: useful, but not automatically trustworthy.
Voice agents are only going to get more autonomous and more connected. The convenience is real. So is the fact that anything the agent can read, someone else can try to write.
Sources
- Zero-Click Calendar Exfiltration Reveals MCP Security Risk in 11.ai — Repello AI: https://repello.ai/blog/zero-click-calendar-exfiltration-reveals-mcp-security-risk-in-11-ai
- ElevenLabs — ThreatNG Security glossary: https://www.threatngsecurity.com/glossary/elevenlabs
- Inside p1bot: A Vishing Platform Weaponizing ElevenLabs — Mirage Security: https://www.miragesecurity.ai/blog/inside-p1bot-vishing-platform-weaponizing-elevenlabs
- 11.ai integrations — ElevenLabs Documentation: https://elevenlabs.io/docs/conversational-ai/legal/11-ai-integrations
- Zero Retention Mode — ElevenLabs Documentation: https://elevenlabs.io/docs/agents-platform/legal/11-ai-integrations
- Ghostcommit disclosure — ASSET Group: https://asset-group.github.io/disclosures/ghostcommit/
- HalluSquatting research — TheHackerNews: https://thehackernews.com/2026/07/new-hallusquatting-attack-could-trick.html