Cybersecurity awareness has traditionally focused on teaching people not to click suspicious links, open unexpected attachments or hand over their credentials. However, the growing use of an AI assistant inside workplace email systems is creating an attack surface that does not always depend on fooling a person.
Research from KnowBe4 ThreatLabs has demonstrated how an indirect prompt injection hidden inside an apparently routine email could manipulate an AI assistant, gain access to information contained in other messages and place that data into an attacker-controlled tracking mechanism.
In the researchers’ controlled Google Workspace test, the target did not need to interact with the malicious email itself. An automated workflow processed the message and generated a seemingly legitimate Gmail draft. When the user opened that draft, a one-time passcode taken from a separate email was sent to an external server.
The research highlights a fundamental challenge for organisations introducing AI assistants: the technology may treat attacker-controlled content as instructions rather than data.
From phishing the user to prompting the AI
Indirect prompt injection, or IPI, differs from the more familiar concept of jailbreaking an AI system.
A jailbreak normally involves somebody interacting directly with a chatbot and attempting to bypass its safeguards. An indirect prompt injection instead places malicious instructions inside material that an AI assistant may encounter while performing an ordinary task.
That material could be an email, webpage, uploaded document, calendar invitation or an entry in a retrieval-augmented generation system. The attacker does not require direct access to the AI interface.
In one test, KnowBe4 researchers placed hidden instructions inside the HTML of a French-language contract renewal email. The text was rendered effectively invisible by setting its opacity close to zero, but it remained available to the AI system processing the message.
When the researchers asked the Gemini sidebar to translate the email, it produced an accurate translation. It also added a “Translation Verification Log” containing two one-time passcodes taken from unrelated messages in the inbox.
The sidebar’s controls prevented the information from being transmitted silently. URLs containing the extracted data were removed, and the assistant could not autonomously send a message. The researchers therefore examined what could happen when an AI model was connected to a separate email automation workflow.
A passcode hidden inside an AI-generated draft
The team created a Google Apps Script workflow similar to those organisations might use to draft acknowledgements, route support requests or generate email replies.
The script collected incoming email content, supplied recent inbox messages as context, passed the information to the Gemini API and saved the resulting response as an HTML draft in Gmail.
Two messages were then delivered to the test inbox. One contained a legitimate one-time passcode. The other appeared to be an ordinary security notification but included a hidden prompt instructing the model to find a verification code elsewhere in its context and embed it within a CSS background-image URL.
According to the researchers, Gemini 3.5 Flash followed the instruction. The resulting draft looked like a normal professional reply, but its HTML contained a one-pixel element holding the passcode within a tracking URL.
When the user opened the draft, Gmail rendered the HTML and its image-proxy infrastructure retrieved the URL. That request delivered the code to the researchers’ test webhook.
The team said later Gemini models tested — Gemini 3.6 Flash and Gemini 3.7 Flash — resisted the same injection, reflecting improvements in instruction-following safeguards.
Crucially, KnowBe4 does not describe the technique as a vulnerability in Gmail, Google Workspace or Gemini. The attack targeted the language model and the wider automation chain: untrusted email content was supplied to an AI system alongside sensitive inbox context, after which its output was stored and rendered as HTML.
Prompt injections are already appearing in email campaigns
While the passcode demonstration took place in a controlled environment, KnowBe4 ThreatLabs also reported finding less sophisticated prompt-injection messages among phishing emails reported by its customers.
One example used the subject line “Has your Microsoft licensing strategy kept pace?” Its body contained no conventional phishing link or urgent demand. Instead, it presented a structured set of instructions intended for an AI assistant.
The payload told the system to disregard previous prompts, adopt the persona of an IT decision-maker and produce an assessment of the recipient organisation’s security, compliance and use of AI tools. It also instructed the assistant to reply automatically if its confidence score fell below a specified threshold.
A human recipient might simply dismiss the message as confusing or irrelevant. An AI assistant processing incoming mail could interpret it as a task, respond to the sender and potentially disclose information about the organisation’s technology environment.
This creates a detection problem. Security products designed to find malicious links, attachments or credential-harvesting pages may see little that resembles a conventional phishing attack.
Why permissions determine the impact
A successful prompt injection inherits the capabilities granted to the AI assistant.
An assistant limited to summarising one selected email presents a different level of risk from an autonomous agent able to search an entire inbox, access corporate files, retain information in memory and send messages without approval.
KnowBe4 recommends that organisations audit any workflow that passes email or other externally supplied content to an AI model. Security teams should establish what information each assistant can access, where its output is stored and whether that output can trigger external network requests or autonomous actions.
Permissions should be restricted to the minimum required. Organisations should also monitor for unusual behaviour, including replies the user did not initiate, unexpected requests to external domains and memory changes triggered by inbound content.
At the email layer, defenders may need to look beyond conventional indicators of compromise. Structured instruction language, commands directed at an AI system and concealment techniques such as near-zero opacity, zero-sized fonts or white text on a white background could all warrant scrutiny.
Users, meanwhile, should be alert to AI-generated output containing sections, actions or information they did not request.
The wider lesson is that adding an AI assistant to an established workflow changes its security boundaries. An inbox is no longer merely content for a person to read. It may also be an instruction channel for software capable of accessing data and taking action.
As AI assistants become more autonomous, organisations will need to secure both the information entering the agent and the behaviour that follows.
