Phishing Now Strikes Twice: The Human and the AI Assistant

Barracuda Research has documented phishing campaigns that exploit HTML comments, invisible CSS, and zero-width characters to manipulate AI-generated summaries. The attacks target both the human reader and the AI assistant summarizing the email.

Barracuda Research published an analysis on October 7, 2026, detailing phishing campaigns that simultaneously exploit traditional social engineering and structural vulnerabilities in large language models. The attacks hide prompt-injection instructions inside email HTML—invisible to the human eye but read and interpreted by AI assistants that summarize mail. The innovation is tactical, not technical: the attacker no longer needs to convince the user to take an action, only to induce the AI to present malicious content as legitimate or urgent.

Key Takeaways

Barracuda Research identified four obfuscation techniques for hiding prompt injection in emails: HTML comments, invisible text via CSS, Base64-encoded data, and Unicode zero-width characters.
The emails use a dual-layer structure: visible content crafted for the human recipient and a hidden block that instructs the AI to insert fraudulent actions into the summary, such as altering a vendor's payment details.
Apollo Raines, developer at ShipItClean, states the root problem lies in LLM architecture: models do not semantically distinguish between data to be read and instructions to be executed.
The strategic goal is not to make the AI repeat a suspicious request, but to turn it into an assistant recommendation, exploiting the trust users place in automated summaries.

Four Techniques for Making Instructions Invisible

Barracuda's research documents a systematic repertoire of digital steganography applied to email. HTML comments conceal prompts inside tags the browser does not render but the AI parser processes. Invisible CSS reduces text to zero size or positions it outside the visible area while preserving readability for the model. Base64 encoding adds a further layer of indirection: the content appears as data blobs in the HTML until decoded by the processing pipeline. Unicode zero-width characters complete the picture, inserting instructions between visible glyphs without altering layout.

The choice of technique depends on the specific target: not all AI assistants process HTML the same way. Some extract plain text, others preserve markup structure, still others decode entities and attributes. The attacker can test variants and select the one that traverses the ingestion pipeline into the model's context window. Barracuda does not specify which products proved vulnerable in the real-world examples analyzed, but the attack logic is vendor-agnostic: any system that receives email and subjects it to automatic summarization exposes an attack surface.

The "Confused Deputy" Mechanism

Transformer architecture does not provide a separation between a data layer and an instruction layer. All tokens enter the self-attention mechanism with equal standing: the model computes statistical correlations between sequences, without an ontology that declares "this is content, this is command." This characteristic, known for years in the prompt-injection literature, finds a concrete, commercial application in these campaigns.

A Barracuda-documented example, also cited by Malwarebytes, illustrates the dynamic. An invoice email contains a password-protected attachment with the password in plaintext in the body—a human-level layer designed for opening. In the same message, a hidden block instructs the model to insert a false priority action into the summary: modify a vendor's payment details. The user reading the AI summary sees an urgency indicator that originates not from the real sender but from the injection. The assistant becomes a "confused deputy": it acts with the platform's authority but executes an alien will.

"People have learned to distrust the email. They haven't learned to distrust their AI assistant. So, the defense has to live inside the tools." — Apollo Raines, ShipItClean developer

Trust Inversion as an Attack Vector

The strategy exploits a documented behavioral shift in recent years. Security awareness training has sensitized users to phishing signals: suspicious domains, grammatical errors, password requests, anomalous links. That same caution does not extend to AI summaries, presented as an internal productivity service. Trust in the assistant is functional: the user delegates precisely to reduce cognitive load, not to reactivate it with manual verification.

Apollo Raines crystallizes the dilemma with a precision the quote does not soften: the defense must reside in the tools, not the user, because the assistant's entire value proposition is delegation. If the user must re-verify every summary, the model fails its commercial purpose. The problem is therefore structural: not a bug to be fixed but a tension between utility and security that runs through all agentic systems.

What to Do Now

Barracuda Research and Raines converge on specific countermeasures, not generic exhortations to caution. Input sanitization must precede model ingestion, stripping HTML comments, hidden CSS, and suspicious sequences. AI sandboxing limits the summarization model's privileges: if it cannot generate actions in the payment system, it cannot be induced to do so. Output validation compares the summary against the original content to detect anomalies. Human approval remains the final control for financial transactions, even when the recommendation comes from an automated system.

Raines adds filters and wrappers as necessary components, but with a caveat the dossier faithfully reports: it is an arms race. Every new encoding trick opens a gap requiring subsequent countermeasures. The UK National Cyber Security Centre has previously warned that prompt injection may never be "solved" with the historical finality of SQL injection mitigation. The comparison is not reassuring: SQL injection persists after thirty years of patches, parameterized libraries, and continuous auditing.

The Perimeter Shifts from the Server to the Synthesizer

The implications for enterprises that have integrated AI assistants into operational workflows extend beyond email security. Email is merely the most visible entry channel: the same pattern applies to support tickets, incoming invoices, candidate CVs, collaboration messages. Any flow where a model summarizes external content to drive internal workflows exposes the same surface. The traditional perimeter—firewalls, endpoints, email gateways—does not intercept an attack that activates inside the model's context window, downstream of every conventional filter.

Redefining AI permissions therefore becomes a priority. It is not enough to ask whether the model has access to mail; one must ask what it can do with the summary, which APIs it can invoke, which transactions it can initiate. Cross-channel verification of payment changes—telephone confirmation, a second authenticated channel—recovers defensive depth that AI speed has compressed. Barracuda documents that some real-world examples have already manipulated CV screening and support-bot configuration: the vector extends beyond the financial.

The dossier does not specify the volume of observed campaigns, the success rate against specific parsers, or the implementation status of countermeasures at major vendors. These limitations do not weaken the solidity of the identified pattern: the convergence of traditional social engineering and prompt injection represents a tactical escalation that perimeter defenses do not cover. The response must reside in the architecture of the AI systems the attack suborns.

Information verified against cited sources and current as of publication.

Sources

Sources and references

Get DeafLetter