RAG oversharing: why AI assistants leak data

There's a scene that repeats in nearly every demo of an AI assistant for companies: someone types a question, the system searches the internal documents and answers in seconds with impressive precision. The demo convinces, the project starts. And a few months later, someone in support asks a routine question and receives, along with the answer, a payroll figure they should never have seen.

It isn't an exotic bug, and it doesn't take a sophisticated attacker to trigger it. It's the direct consequence of how semantic search works when nobody has thought about permissions from day one. It's worth understanding the mechanism, without jargon, because if you're evaluating an AI assistant for your company this is the first thing you should look at.

What happens to your documents when they enter a semantic search index

For an AI to answer questions about your documents, they first have to be turned into something searchable by meaning, not just by exact words. That process chops each document into fragments and turns each fragment into a vector: a long list of numbers representing what that text is about. When someone asks something, their question is also turned into numbers, and the system retrieves the fragments most similar to it. This architecture is called RAG (retrieval-augmented generation) and it's the basis of practically every corporate knowledge assistant on the market.

The problem is born in a detail that's easy to overlook. The original document lived in a folder with permissions: only HR could see it, or only management. But the vectorised fragment is a different thing. It lives somewhere else, in a vector store, and unless someone designs it deliberately, it doesn't carry with it the information about who could see the original. From there, the search engine does its job: find what is most relevant to the question. Which is not the same as what that person has permission to see. They are two different criteria, and the second one doesn't come as standard.

The usual patch: filter after searching

Most implementations try to solve this in the application layer: first they retrieve the most relevant fragments and then, with the results already in hand, they discard the ones the user shouldn't see. Retrieving first and filtering afterwards is a documented and recurring failure pattern in the industry; it isn't the anecdote of one careless implementation.

Why does it fail? Because that filter is application code, and application code has bugs, edge cases and metadata somebody forgot to fill in. It only takes a connector that indexes one folder too many, a document with content mixed from two departments or a badly written condition for a restricted fragment to reach the model, and from there the answer. And there's something worse: even when the filter works, the restricted content has already travelled from the database to the application. The door was open; you were just trusting the last guard to look properly.

A hypothetical example (but a perfectly plausible one)

Imagine someone in operations asks: «How is the weekend shift bonus calculated?». In the vector store there's a fragment of the department's full salary table, because at some point someone uploaded the spreadsheet to the wrong shared folder, or because the connector indexed more than it should. Semantically, that fragment is the most relevant thing there is for that question. If the permission wasn't checked at retrieval time, the answer arrives with names and figures that person should never have seen. Nobody has hacked anything: the system has done exactly what it was built to do.

How widespread is this problem? I'll be honest: the specific figures doing the rounds come from scattered sources, not audited independent studies, so I'm not going to give you a percentage as though it were a verified fact. What can be said is that industry analyses point consistently in the same direction: leaks between departments or between clients in RAG systems are a widespread problem, not a marginal one. Widespread enough that it should be the first question you put to any vendor.

How it's done properly

The alternative holds no conceptual mystery, though it demands considerably more engineering work. It comes down to three decisions that have to be taken at the beginning of the design, not at the end:

The permission is checked at retrieval, not afterwards. The access filter is part of the query itself, enforced at database level. It isn't an «if» in the application code: it's the database refusing to return what doesn't correspond.
Isolation is enforced in independent layers. The database and the vector store each apply their own separation, so a failure in one layer doesn't compromise the other. And it's verified with automated tests, not with trust.
Genuinely sensitive data doesn't even enter the game. Payroll and employees' personal data are not vectorised and do not pass through the semantic search: every access to them is checked explicitly on the server, query by query.

That's how Savia has been built from day one: each department has its knowledge space isolated at database level, isolation between companies is verified across two independent layers with automated tests, and employees' sensitive data lives outside the semantic search pipeline entirely. Not through being cleverer than anyone else, but through having started with the permissions question instead of leaving it until the end.

If you're evaluating AI assistants, don't ask «does it have permissions?»: they will all say yes. Ask where they are applied. The answer separates systems designed for a company from demos that only aspire to impress in a meeting.