RAG security: how retrieval augmented generation gets attacked

Retrieval augmented generation, usually shortened to RAG, has become the standard way to give an AI product access to your own data without retraining a model. Ask a question, the system retrieves relevant documents from a knowledge base, and the model uses them to generate an answer grounded in your actual information. It works well, and that is exactly why so many AI startups have built it into their product without stopping to ask how it can be attacked.

Most teams that deploy RAG focus their security thinking on the obvious entry point, malicious things a user might type. That is only one part of the picture, and often not the most dangerous one. This article covers how RAG systems actually get attacked across the whole pipeline, using real documented research and incidents, and what genuinely helps.

Why RAG creates a new attack surface

A RAG system has three stages, and each one can be attacked differently. First, documents are processed and stored as vector embeddings in a knowledge base. Second, when a query comes in, the system retrieves the documents most relevant to it. Third, those retrieved documents are inserted into the prompt as context for the model to generate an answer from.

The core problem is that most RAG systems have no reliable way to distinguish a trusted instruction from content that simply happened to be retrieved. Whatever sits in your knowledge base gets treated as legitimate context the moment it is retrieved, and the model has no innate way of knowing that a document was planted there specifically to manipulate it.

Attacking the knowledge base, corpus poisoning

This is the attack that should concern most AI startups the most, because it is both effective and quiet. Corpus poisoning involves getting malicious content into the documents your RAG system retrieves from, so that when a relevant query comes in, the poisoned content gets pulled into the model's context and shapes its answer.

What makes this genuinely alarming is how little poisoned content it actually takes. Academic research has demonstrated that inserting as few as five malicious documents among a knowledge base containing millions can achieve a success rate above ninety per cent for a targeted attack. You do not need to compromise a large share of your data. A handful of carefully crafted documents, seeded into wherever your system ingests content from, can reliably manipulate specific answers while the rest of your knowledge base remains completely normal.

The practical routes for this are wherever your system pulls in new content without careful review, a shared document store, a web page your system scrapes or references, a user submitted file, or any pipeline where external parties can get content into your index. Once poisoned content is embedded and stored, it can quietly influence every future query that happens to retrieve it, sometimes for a long time before anyone notices, because a poisoned answer often looks like a plausible answer.

Attacking the retrieval and embedding layer

The retrieval step itself has its own vulnerabilities, separate from what is sitting in the knowledge base.

Embedding inversion is a documented risk where an attacker who gains access to the vector representations stored in your database can recover a meaningful portion of the original text those vectors were built from. This matters because teams sometimes assume that storing data as embeddings rather than plain text is inherently protective. It is not. Embeddings can leak more of the original content than most teams expect.

There is also a tenant isolation problem specific to how most vector databases actually work. Many vector databases support separating different customers' data through namespaces or metadata filtering, but that separation is typically enforced at the application level rather than as a built in, guaranteed property of the database itself. In practice this means the isolation between customers is only as strong as the code your team writes to enforce it. If a query is not properly scoped with the right permission filters, one customer's data can end up retrieved into another customer's results, not because the vector database failed, but because the enforcement sitting on top of it had a gap.

Attacking through retrieved content, indirect injection at scale

RAG is one of the clearest real world settings for indirect prompt injection, the pattern where a malicious instruction is hidden inside content the AI reads, rather than typed by the user. In a RAG system, every retrieved document is a potential vehicle for this, because the retrieved text goes straight into the model's context with little structural separation from a genuine instruction.

This is not a theoretical risk. A documented vulnerability in Slack's AI features combined indirect prompt injection with Slack AI's retrieval based functionality, showing that this exact pattern reaches production systems from major, well resourced companies. A separate documented issue involved memory poisoning in ChatGPT, where manipulated stored content influenced later behaviour. These are not edge cases invented by researchers. They are real incidents in real deployed systems, and the pattern they share, retrieved or stored content quietly steering the model's behaviour, applies to any RAG system that treats what it retrieves as automatically trustworthy.

A particularly patient version of the attack

One further pattern is worth understanding because it is unusually hard to catch. A poisoned document does not have to act immediately. Researchers have described payloads that sit dormant, worded so that they only trigger a change in behaviour under a specific future condition, meaning a poisoned entry can sit unnoticed in a knowledge base for a long time before it ever produces a visible effect. This turns RAG poisoning into something closer to a slow supply chain compromise than a one off attack, which is exactly why catching it after the fact, rather than preventing it at ingestion, is so much harder than it sounds.

What actually helps

There is no single fix, and current research is honest that inference time defences alone are not sufficient against sophisticated poisoning. Real protection comes from layering controls across all three stages of the pipeline.

• Control what gets into your knowledge base. Treat document ingestion the way you would treat code entering your repository. Know the source of every document, review or restrict who and what can add content to your index, and be especially careful with any pipeline that ingests from external or user supplied sources automatically.
• Separate retrieved content from instructions structurally. Do not let retrieved text sit in the prompt in a way that can be confused with a system instruction. The clearer this separation, the harder a poisoned document is to weaponise.
• Enforce tenant isolation properly, and verify it. Do not rely on the vector database's namespace features alone. Confirm that every query is scoped with permission aware filters in your own application code, and test that one customer's query genuinely cannot retrieve another's data.
• Monitor for anomalous retrieval and output patterns. Because poisoning can be subtle and can lie dormant, ongoing monitoring for unusual answers or unexpected retrieval patterns catches things a one off review will not.
• Limit what a RAG fuelled agent can actually do. Where a RAG system feeds an agent with access to tools, treat retrieved content as being able to influence tool selection and action, not just text generation. The same layered controls that limit agent privilege in general apply here with extra force, because retrieval poisoning combined with tool access is significantly more dangerous than retrieval poisoning that only affects a chat answer.
• Periodically audit your knowledge base. Since a small number of poisoned documents can cause disproportionate harm, occasional review of what has actually accumulated in your index over time is worth the effort, particularly for sources that update automatically.

Why this matters for enterprise buyers

If your product uses RAG, and increasingly most AI products do, this is precisely the kind of question a knowledgeable enterprise security team will now ask about. How do you control what enters your knowledge base, how do you keep one customer's data separate from another's in retrieval, and how do you prevent retrieved content from being treated as instructions. A vague answer here signals the same thing a vague answer on prompt injection does, that the team building the product has not thought carefully about how their own architecture can fail.

The honest takeaway

RAG makes AI products genuinely more useful, and it also introduces a real, demonstrated attack surface that most teams have not fully considered, spanning the knowledge base, the retrieval and embedding layer, and the way retrieved content reaches the model. The attacks are not hypothetical. They have been demonstrated in research with alarmingly high success rates from a small number of poisoned documents, and they have already appeared in production systems at major companies.

If your product retrieves from a knowledge base, treat that knowledge base as a security boundary, not just a data store. Control what goes in, separate it structurally from instructions once it comes out, and verify your tenant isolation rather than assuming it. That combination is what turns RAG from an unexamined risk into something you can speak to with genuine confidence in front of a buyer.