Artificial Intelligence / Architectures
What Architecture Should You Choose for Your Artificial Intelligence Project?
At Microsoft, on-site, in a hybrid setting, or next to the machine: first the criteria, then a tentative answer.The question we’re asked is, “ cloud or local?” It’s almost always the wrong question. The right questions are: Where is your data, who has permission to access it, how quickly must a response be provided, and what happens when the connection goes down? Seven questions are enough to guide the way, and one possible answer is to build nothing at all.
Three Mistakes People Make Before Writing the First Line of Code
They aren't technical. They have to do with the order in which the questions are asked, and the choice of architecture almost always comes too early.
Deciding on accommodations before knowing the details
We tend to choose “on-premises” out of caution or “online” out of habit, before even looking into where the documents are stored, who has access to them, and which ones are actually not confidential at all. The location is determined by the data, not the other way around.
Sensitivity is determined on a document-by-document basis.
Count the servers, forget about operations
A platform on your premises is a server with graphics processors, but above all, it’s someone who updates the models, monitors costs, and steps in on a Sunday. That position doesn’t appear on the hardware quote.
Rent for the space is paid monthly, not all at once.
Believing that online services are always cheaper
For low volumes, yes. Once you exceed a certain number of requests per day, or when millions of pages need to be re-read each month, local processing becomes less expensive. This threshold is calculated over a three-year period; it can’t be guessed.
The tipping point exists, and it can be quantified.
Four Ways to Implement the Same Solution
Whether it’s an assistant for your documents, invoice processing, or end-of-line inspection, there are four possible approaches. Each has its advantages, limitations, and situations where we recommend it. The detailed pages cover the rest.
OnlineAt Microsoft
The data is already in Microsoft 365. The features come with the licenses. No hardware to buy, nothing to maintain—a first implementation in just a few weeks.
Cost is based on usage and increases with volume. The service must be able to process the data. Latency depends on the connection.
Assistants in Teams, search for SharePoint, email filtering, and customer service. Everything that’s already available in your tenant.
On-premisesAt your facility
Nothing leaves the premises: not the documents, not the requests, and not the template. The cost per request becomes negligible once the equipment has been paid for. Large volumes remain on your premises.
A server with graphics processors, its power supply, its cooling system, and the operations that go along with it. Models are evolving quickly; we have to keep up with them.
Confidential data sets, source code, regulated data, large volumes of documents, or a contract that prohibits the use of an online service.
HybridPartly at Microsoft, partly at your location
Identity, the " Teams ," and the interface remain with Microsoft; document search, sensitive data, and inference remain on your premises. The best of both worlds—when it’s properly separated.
Two locations to operate, a border to secure over the long term, and a connection between the two that has become critical.
The most common scenario: an assistant in Teams whose corpus and model never leave datacenter. Or the processing of documents where the sensitive portions remain on-site.
EdgeNext to the machine
The decision is made on the spot, in a matter of milliseconds, with or without a connection. The images and measurements do not travel over any network.
Rugged equipment in a harsh environment, remote management, and templates that need to be updated at each site.
Cameras on a line, sensors on equipment, industrial sites, or remote locations that cannot rely on the Internet.
The Enterprise Platform
With the third prototype, the question changes: it is no longer “where does this case work,” but “what common, governed, and supervised infrastructure will multiple services share?” This is a fifth answer.
The criteria, line by line
Eight criteria, four models, and what each box means to you. There is no “winning” column: the one that carries the most weight in your situation is the deciding factor.
| Criterion | At Microsoft | At your facility | Hybrid | Next to the machine |
|---|---|---|---|---|
| Data Sensitivity | Processed in the Microsoft cloud; data location in Switzerland to be verified | Nothing comes out | Sensitive information remains on your premises | Nothing leaves the site |
| Data volume | Pay-as-you-go: increases with volume | Unlimited once the equipment has been paid for | The bulk of it remains local | Processed on-site as it flows |
| Latency | Depends on the connection; a few hundred milliseconds | Internal Network | Depending on the party being asked | A few milliseconds |
| Connectivity | Always Essential | Internet-independent | The connection between the two becomes critical | Continue when the connection is lost |
| Inference Cost | The cheapest option for small volumes | The lowest price beyond a certain number of requests | The threshold is calculated based on the local portion | Equipment sized for the site |
| Integrations | Natives with Microsoft 365, Dynamics 365, Power Platform | To be integrated with your systems | Both, with a line to hold | Toward Industrial Systems and theERP |
| Day-to-Day Operations | Powered by Microsoft | Entirely yours, or entrusted | Shared | Remotely, site by site |
| Time frame for the first case | A few weeks | A few months, including equipment | According to the local party | Depending on the location and the sensors |
The mechanisms described here are taken from Microsoft documentation and Dell Technologies, and are double-checked before each proposal. No prices are listed here: they change every quarter, and a pricing error is the kind of mistake that makes people stop trusting everything else.
Seven questions, one suggested answer
Answer as you would during an initial meeting. The result provides guidance; it doesn’t define anything: it identifies the architecture we should examine first, the pitfall associated with it, and the page to read next.
What is your use case?
Where is the data located?
Can data leave your infrastructure?
What volume needs to be processed?
Is a real-time response necessary?
Does the system need to work without an Internet connection?
Do you already have servers with graphics processing units?
Answer the seven questions
The conclusion becomes clear once all seven answers have been provided. It identifies the architecture we should study first, the first pitfall, and the page to read next.
Identify the use case first
Until the use case is defined, no architecture can be chosen: the solution is derived from the problem, never the other way around. A half-day with the business stakeholders is often enough to identify three candidates and narrow it down to one.
The first pitfall: starting with the technology you want to try.
Identify the right use casesNext to the machine: Edge AI
Without an Internet connection, or when a decision must be made in a matter of milliseconds on the machines, processing must take place on-site. This requires robust on-site equipment, managed remotely, and models updated from the central office when a connection is available.
The first pitfall: underestimating the environment—dust, heat, a lack of IT staff—and the cost of updating each site.
View the Edge AI ArchitectureOn-premises: on-premises artificial intelligence
Your data cannot leave the system: the template, documents, and calculations all run on your datacenter. Regulatory requirements take precedence over everything else; they are the only non-negotiable factor.
The first pitfall: buying the server before deciding who will operate the platform, update the models, and monitor costs.
View Private ArchitectureAn enterprise platform
Whether it’s multiple use cases, high volumes, or existing computing infrastructure: the question is no longer “where does this use case run?” but “what shared infrastructure will multiple departments use?” Governance, monitoring, and costs are decided once and for all.
The first pitfall: letting each department build its own prototype, only to end up with four platforms and no operational use.
View the enterprise platformHybrid: The interface is at Microsoft; the data is on your premises
Some data can be moved off-site, while other data cannot, or it may be distributed across multiple locations. Identity, Teams , and the interface remain with Microsoft; sensitive or large-volume data stays on your premises. This is the most common setup we implement.
The first pitfall: a blurred line between the two, which shifts as demands change and ultimately fails to protect anyone.
See Hybrid ArchitectureAt Microsoft, online—and often, what you already have is enough
Your data resides in Microsoft 365, it can be processed there, and there’s no need for a real-time response. There’s no reason for an infrastructure project here. Copilot within your licenses, an agent Copilot Studio , or a service Azure will handle this—provided you first clean up the access rights.
The first pitfall: opening a wizard on a SharePoint system whose shared folders have never been reviewed. The wizard doesn’t create leaks; it reveals the ones that already existed.
View Architecture OnlineThe result is for reference only. The final decision will be based on your actual data, your integrations, and your operating budget during your assessment.
What Seven Questions Cannot Decide
There are three things to consider when building your assessment, based on your numbers. Ignoring them is the surest way to make choices that look good on paper but don't work in the machine room.
Where are the systems that consume the output located?
An assistant who needs to open a file in Dynamics 365, post toERP , or send an alert to Teams drives the architecture toward the location where these systems reside. The location of the data alone is not enough.
The threshold beyond which your office space becomes less expensive
Number of requests per day, pages reviewed per month, minutes of voice recording: these three figures, when compared to the cost of equipment and its operation, establish a threshold. We’ll calculate it right here in front of you.
Who's in charge of the platform on Sundays?
Any infrastructure on your premises or at a site requires someone to update, monitor, and maintain it. We can handle that; it just needs to be decided and costed out before you make a purchase.
Each Architecture, in Detail
Five pages outline the benefits, limitations, use cases that justify them, and the technologies that power them.
At Microsoft, when it's the right answer
Copilot, Copilot Studio, Microsoft Foundry, the services Azure : what happens with your licenses, and when they’re enough.
View the page On-premisesOn-premises: data, models, and computations on your servers
When nothing should be released. Azure Local, Foundry Local: open-weight models on Dell PowerEdge servers with graphics processors.
View the page HybridThe Microsoft interface, sensitive data on your premises
The most common classification in practice, and the boundary that must be maintained over time.
View the page EdgeEdge AI: Where Data Is Generated
Factory, warehouse, store, remote location: decisions made on-site, and operations when the connection goes down.
View the page PlatformAn infrastructure shared by multiple departments
Multiple models, multiple cases, one governance and oversight framework: the response to the third prototype.
View the page Your ProjectFrom Planning to Day-to-Day Operations
How to really choose an architecture: based on your data and with your users, before buying anything.
View the pageLet's talk about your architecture
Your responses, your real-world data, and a choice that stands the test of time
Send us your seven answers and two orders of magnitude: the number of documents or queries, and what must not be disclosed. We’ll get back to you with the architecture we would consider, the cost threshold, and what would need to be measured to validate it.
What we offer is a meeting with the engineers who will do the work. The threshold calculation and detailed design are part of the first- assessment. The discussion that precedes them, however, is free of charge.
Renens, Sion, Châtel-Saint-Denis
Microsoft Solutions Partner and Dell Technologies Gold Partner : we have expertise in all four architectures, and that’s what makes consulting possible.
Renens VD +41 21 806 37 15
Sion VS +41 27 552 00 22
Châtel-Saint-Denis FR +41 26 322 59 05