Artificial Intelligence / Use Cases / Private RAG
A private RAG: your documents, your model, your infrastructure
Search a confidential corpus using everyday language, citing the source, without a single word leaving your datacenter.RAG—short for retrieval-augmented generation —involves first identifying the relevant passages in your documents, then having a model generate the response based solely on those passages. A private RAG performs both steps on your premises: the index, the search, the model, and the logs are all hosted on your datacenter, under your control. This use case exists because the online service is off the table—whether due to a contract, confidentiality, or regulations.
Three situations where online service isn't an option
As a general rule, we do not recommend the local option. In these three cases, the question does not arise: the requirement is specified in writing, and it takes precedence over everything else.
The contract with your client prohibits the disclosure of their documents
Whether it’s a firm, an engineering firm, or an industrial subcontractor: the files belong to the client, and the contract specifies where they may be processed. No matter how good the online service is, sending the files themselves would be a breach of contract. Yet teams need to be able to find, in a matter of seconds, what a particular file says about a specific issue.
You don't argue with a clause.
The corpus is the company's asset
A manufacturer’s processes, a laboratory’s results, an engineering firm’s calculation models, an insurer’s claims files. Twenty years of work, contained in thousands of documents that few people can still read. Making them searchable is invaluable; entrusting them to a third-party service is out of the question.
The risk isn't in the use; it's in the exposure.
Regulations or professional confidentiality apply
Patient data, court records, banking data, classified information: a fixed legal or ethical framework determines who can access them and where. The location is not a technical choice; it is the architectural embodiment of that framework.
Architecture follows the law; it does not circumvent it.
What a private RAG does, in everyday language
Two steps, always in this order: research, then write. It’s the first step that determines the quality, and that’s the one we tend to overlook.
The system cuts and indexes your documents on your premises
Each document is divided into sections, and each section is assigned a unique identifier that allows it to be found based on context rather than the exact wording. The index resides on your storage device and updates automatically whenever the documents change.
The system identifies the relevant passages, within the limits of the individual's access rights
For each query, it returns the ten most relevant results—from among those the user is authorized to read, and only those. Entra ID The source of the rights remains.
The model writes based on these passages—and nothing else
He doesn't draw on what he's learned elsewhere; he rephrases what your documents say, and he cites every passage he uses. If there's no relevant passage, he says he doesn't know.
The system compares what various documents say
Two versions of a clause, three reports on the same document, the discrepancy between what the contract says and what the amendment says. That’s where the time savings are most evident.
The system logs every question and every passage read
Who requested what, and which documents were used. This is what makes it possible to respond to an audit and correct the index when an answer is incorrect.
The system does not call any online services
No need to use an online service for fingerprinting, searching, or writing. This can be verified in the network logs, and it is the only valid definition of the word “private.”
A concrete, step-by-step example
An engineer at an engineering firm asks, “What tolerances have we approved for this type of assembly, and in which reports?” The dataset consists of fifteen years of test reports and calculation notes, covered by confidentiality agreements with clients.
What Remains Human
A private RAG makes your documents searchable. It does not determine what is true or what is included in the corpus. Five decisions are left to appointed individuals.
Decisions That Cannot Be Delegated
Recorded in the system file before the system goes live, and reviewed during each quarterly review. A RAG whose database is not maintained by anyone will become outdated within six months.
- What is included in the corpus. The corpus manager decides which documents are indexed and removes those that should no longer be included.
- Verify the source before taking action. A response is a lead backed by evidence. Anyone using it to make a technical decision should open the cited report.
- Handling contradictions. The model flags them; a human makes the final decision and either annotates or archives the outdated document.
- Scope of access rights. The company— Entra ID —determines who has access to which projects; this is never determined by the system.
- Choosing and replacing the model. Tested based on your questions, compared, and decided upon every few months.
The data used, and what they require
A RAG is only as good as its index, and the index is only as good as the documents it contains. Quality is determined here, even before the first graphics processor comes into play.
Sources
- Confidential documents: reports, contracts, files, memos, plans—on your file-sharing platforms, in SharePoint Server, or in a document management system.
- Metadata: client, project, date, version, status. It filters the search even before the content is considered.
- Entra ID : the groups and permissions that define the scope of each search.
- Business systems: eERP, EDM, Dynamics 365, accessed via their interfaces when required.
What they must have
- Machine-readable documents. A report that has been scanned at an angle, a table as an image, or a map without text cannot be indexed. Character recognition is part of the project, but it has its limitations.
- One status per document: active, replaced, archived. Even without it, the template also lists what has been corrected.
- One owner per corpus, named, who responds when an answer is incorrect.
- Figures that reflect the truth, recorded before indexing.
- An initial, limited corpus: one domain, a few thousand documents, twenty users. The rest will come after the first review.
Where the RAG works, and which one we would study first
In this case, the issue is almost settled by its very definition. A variation does exist, and it is more common than one might think.
At Microsoft
Excluded by definitionIf an online service were possible, it would be an assistant for your documents, not a private RAG. The next page discusses this.
At your facility
RecommendedThe index, search, model, interface, and logs in your datacenter. Nothing is out in the open. This is also the architecture that requires the most maintenance.
Partly at Microsoft, partly at your office
PossibleTeams for the interface and Entra ID for identity at Microsoft; the index, search, and model are hosted on your premises. Only the question and the answer are transferred—never the documents.
Next to the machine
Possible, in a remote locationThe same system, at a location without a reliable connection: a construction site, a ship, or a mountain facility. The index is synchronized when the connection is restored.
Why at your office, and what makes the hybrid model the right choice
Three criteria are decisive for this use case. The first settles the matter; the other two partially reopen it.
| Criterion | What tips the scales | Toward what |
|---|---|---|
| The provision prohibiting the release of data | Contracts, regulations, professional confidentiality, internal policies. This requirement is not technical and is non-negotiable: the index, the search, and the model must remain on your premises. | On-premises |
| The Interface and Identity | If your teams work in Teams, asking a question there and getting an answer—without any documents changing hands—is still the norm. This is the hybrid model, and it’s evident in the news. | Hybrid, if the text allows the question and answer to be transmitted |
| The volume and frequency of requests | Whether it’s re-reading millions of pages every night or handling hundreds of requests per minute, local computing is becoming cheaper than an online service would be. That’s one more argument in favor of local computing, even without any constraints. | On-premises, and sometimes a corporate platform |
One point to clarify with your data protection officer before any hybrid split: exactly what the policy prohibits. Some policies prohibit the document from leaving the organization; others prohibit the request itself—which may contain data—from passing through a third-party service. In the latter case, only a fully on-premises solution is acceptable.
What the RAG connects to
A self-contained system serves only to respond. Its value comes from what it reads and what it triggers in the systems you already have—without copying them.
Infrastructure: Here, it is necessary
For this use case, privacy requirements make local execution mandatory. We therefore also scale the necessary computing infrastructure —and then maintain it.
The calculation
A Dell server ( PowerEdge ) with one or more NVIDIA graphics processors for inference and feature extraction. The number and type are determined by the number of concurrent users, the model size, and the acceptable response time, as measured in the prototype. For more than one team, the XE series, designed for multiple accelerators.
Data Storage
A document corpus and its index consist of millions of small files and fragments. Dell storage is chosen for this profile—file or object—and for the speed at which the model must read from it.
The foundation and the language model
Depending on the situation: Azure A solution for hosting Microsoft tools on-premises, using Foundry Local or Edge RAG—currently in preview at Microsoft for this version—; or a pipeline built on open-source components, a vector search engine, and an open-source, version-locked model. We choose the approach on a per-project basis and implement it.
Day-to-Day Operations
Update the model, reindex, monitor latency and storage space, back up the index, and roll back. This can be done by us as part of our managed services, or by your team—which we train. What doesn’t work is not deciding this before you buy.
What you measure determines what you buy
The Dell server sizing process— PowerEdge —cores, memory, disk access, accelerators, growth—applies word for word to a private RAG. The server is ordered after the prototype is built, never before.
How We Carry Out the Project
Don't buy anything until the prototype has been tested on your actual documents. That's the process that prevents the server from sitting idle, waiting to be used.
The Framework: The Text, the Corpus, the Questions
Exactly what the contract or regulations prohibit. The initial dataset, its readability, its owner, and its rights. The twenty questions teams actually ask. And calculating the cost threshold when considering a hybrid approach.
The prototype, based on your documents, using existing or loaned equipment
A representative sample from the corpus, a prototype, about 20 users, three to four weeks. We measure the accuracy of responses to the identified questions, latency, and how users interact with the system.
This is where the project can end, and that's fine.
Architecture and Hardware
Server, graphics processors, storage, and backup, all sized according to the prototype specifications. The platform and model have been selected and configured. Prepared prior to delivery and connected to Entra ID and your systems.
Full Indexing and Validation
The entire indexed corpus, the applied statuses, the tested permissions—a person without permissions for a project must not be granted any. Gradual rollout, team by team.
Day-to-Day Operations
Reindexing, updating the model, backing up the index, and reviewing flagged responses and identified inconsistencies. Performed by us or by your team.
Estimated timeline: three to five months from the initial assessment to going live, including equipment delivery time, for an initial corpus of a few thousand documents and a single team. This is a rough estimate, not a commitment: measuring progress is part of the job.
Scope and Limits
What a private RAG does not do, and what we do not promise. The location changes the place where the processing takes place, not the nature of the model.
The model is wrong, with the same confidence as online
The quoted passage serves as a safeguard: it allows for verification. The system is designed to generate text only based on the passages it finds and to indicate when it finds none—but a rephrasing can give away a nuance. Human validation remains essential for any binding decision.
The model you'll be able to run is smaller
The largest models on the market require computing power that few companies have in their server rooms. A smaller model, trained on your data, often answers your specific questions better than a large, general-purpose model—but performs less well on everything else. We test it on your questions before drawing any conclusions.
Literature review is more decisive than the model
If the right passages aren't found, even the best model in the world will give the wrong answer. Document segmentation, metadata, statuses, and permissions are what determine quality—and that's a data task, not an artificial intelligence one.
Unreadable documents remain unreadable
Crooked scans, pages without text, tables as images: character recognition helps, but it can’t work miracles. Part of the corpus may remain out of reach, and it’s better to know that at the prototype stage.
The cost is charged monthly, regardless of usage
Equipment, electricity, cooling, licenses, and the time of the people who maintain the platform. We compare these costs over a three-year period to the cost of an online service—where possible—and to a hybrid model. Sometimes an on-premises solution isn’t the least expensive option, and we make that clear.
"Private" is a property that can be verified
No external services are required for fingerprinting, searching, writing, or telemetry. This can be verified in the network logs, and we have your team verify it during commissioning.
Related cases and their corresponding pages
When another use case is the right one, and the infrastructure behind it.
Privately Housed Business Assistant
When part of the corpus can be read at Microsoft and the other part cannot: Teams for the interface; the sensitive data is stored on your premises.
View the case A similar caseAssistant for Company Documents
When the documents are in SharePoint and can be read there: Copilot or an agent, without any hardware.
View the case ArchitectureArtificial Intelligence on Your Premises
What “everything on your premises” really entails—and the technologies that make it work.
View the page InfrastructureDell Servers PowerEdge
The scaling chain and the XE series for accelerated computing.
View the page InfrastructureDell Storage: Millions of Small Files
File and object for the corpus and index, showing what the reduction rates actually entail.
View the page ArticleWhat Really Protects Your Data from Public Models
What a public service does with your requests, what a server guarantees, and when only the local server responds.
Read the articleLet's talk about this use case
Tell us what the text prohibits and what your teams are looking for
The clause, regulation, or directive that precludes online service; the document repository and its current state; the questions teams are asking today that lack quick answers. We’ll revisit the architecture we’re considering, the cost threshold for a hybrid approach, and what a three-week prototype using your documents would help us assess.
What we offer is the opportunity to meet the engineers who will do the work. Assessment — prototyping and sizing are part of the project. The initial consultation, however, is free of charge.
Renens, Sion, Châtel-Saint-Denis
Microsoft Solutions Partner and Dell Technologies Gold Partner. The research, the template, the rights, and the server that hosts them—all handled by the same point of contact.
Renens VD +41 21 806 37 15
Sion VS +41 27 552 00 22
Châtel-Saint-Denis FR +41 26 322 59 05