Private AI/vs Microsoft Foundry
Comparison

Private LLM hosting vs Microsoft Foundry (formerly Azure AI Foundry)

By Samuel Seidel · Published September 9, 2026 · 9 min read

I run GPUwerk, which rents DGX Sparks, so weigh everything below with that in mind. Microsoft Foundry, called Azure AI Foundry until its latest rename, is Microsoft's platform for building and deploying AI applications, model catalogue, evaluation, and agent tooling, layered across models from OpenAI and other vendors. Microsoft has reorganised this part of Azure more than once, so the Foundry-specific claims here stay general rather than guess at product details not checked against Microsoft's current documentation. If your interest is specifically GPT-model access, the OpenAI API comparison goes deeper on the model side; for Azure's own terms, Microsoft's documentation is the source.

Side by side

Microsoft FoundryDedicated DGX Spark (GPUwerk)
Where data is processed Depends on the model deployment and region you configure within Foundry. Microsoft's documentation for the specific model family in question, not a platform-wide statement, is the source to check, since options and defaults vary by which model and deployment type you pick. EU-Central (Hradec Králové), on one dedicated machine. No deployment configuration to get wrong, because there's only the one machine and one region. (advantage)
Who can see your prompts Governed by Microsoft's platform-wide data, privacy, and security documentation, with specifics that can differ by model vendor within the catalogue. Microsoft publishes abuse-monitoring terms for its OpenAI-model deployments (data, privacy and security for Foundry Models sold by Azure); that page now covers Azure OpenAI and the other models Microsoft sells directly; models from other vendors in the catalogue can carry their own terms. No other customer, and GPUwerk only under the DPA. GPUwerk's DPA states GPUwerk "hosts the machine but does not access, read, copy, index, or analyse the content of the controller's instance." No abuse-monitoring or content-scanning layer exists at all. (advantage)
Model choice A catalogue spanning closed models (OpenAI's among them) and a growing set of open-weight models from third-party vendors, plus Microsoft's own model families, all behind one platform. (advantage) Open-weight models only: gpt-oss-120b, Llama 3.3 70B, Qwen3-Coder 30B, DeepSeek, and anything else that fits in 128 GB. If a model in Foundry's catalogue has open weights, the same model may run directly on a Spark; closed models in the catalogue do not.
Speed and concurrency Throughput depends on the specific model and deployment tier chosen; Foundry pricing and quota are generally expressed in tokens and requests per minute rather than a published tok/s figure. A single Spark, single-stream: gpt-oss-120b decodes at 33.5 tok/s and reaches 862.8 tok/s aggregate at 256 concurrent requests (Dendro Logic's concurrency benchmark, run on their own Spark). Raw logs and methodology on GPUwerk's benchmarks page. (advantage)
Pricing model Per token or per deployment unit, varying by model vendor and tier within Foundry. GPUwerk did not fetch a current figure for this page; check learn.microsoft.com's Foundry pricing documentation for your specific model. Per hour, billed per minute: $0.79/hour on-demand, $0.59/hour while stopped and held for you. One rate regardless of model. USD, tax extra, per pricing. (advantage)
Worked cost example See the full arithmetic below the table.
Contracts and DPA Governed by the Microsoft Products and Services Data Protection Addendum, the same document that covers Azure OpenAI. Enterprise agreements are typical at volume. A standard GDPR Article 28 DPA published free at /legal/dpa, no negotiation required. Sub-processor list at /legal/sub-processors states none are engaged for instance workloads. (advantage)
What you operate yourself Nothing at the infrastructure layer; Microsoft manages capacity and serving. You manage prompts, application code, and evaluation/agent pipelines you build on top of Foundry's tooling. Root access, from the browser terminal or over SSH with your own key, to your own container: the model server, any RAG or agent layer, monitoring, and backups. GPUwerk keeps no copy of /workspace, not even for a failed drive, so that responsibility is entirely yours.

The worked cost example

Take a workload that generates 500 million output tokens a month, roughly what a busy internal support bot or a document-processing pipeline running continuously might produce. Two ways to serve it:

Microsoft Foundry. Pricing depends on which model and deployment tier you choose from the catalogue, and Foundry's own pricing pages present that through model-specific tables that GPUwerk did not fetch and verify for this page. Run your own model and volume through Microsoft's current Foundry pricing documentation for a number you can trust.

Dedicated Spark. gpt-oss-120b reaches 862.8 tok/s aggregate at 256 concurrent requests, per Dendro Logic's concurrency benchmark on a single Spark (cited on GPUwerk's benchmarks page). That is 862.8 × 3,600 = 3,106,080 tokens per hour if the node stays saturated at that concurrency the whole time. 500,000,000 ÷ 3,106,080 ≈ 161 hours. At $0.79/hour, 161 × $0.79 = $127.19, before tax.

That Spark figure only holds under the assumption stated: the node busy at 256 concurrent requests for the full 161 hours, back to back. Real traffic arrives in bursts, so a Spark billed for every running minute that sits half-idle can lose ground fast against a per-token price with no idle charge. Put a current Foundry number for your chosen model next to $127.19 and you have the real comparison for your own volume.

Migration path

For any Foundry model served through an OpenAI-compatible chat completions endpoint, a Spark serving a model through vLLM exposes the same shape, so an application that only calls that endpoint moves with a base URL and API key change. Put LiteLLM in front of the Spark and you get a drop-in OpenAI-compatible gateway with request logging, key management, and rate limits, useful for routing between a Foundry-hosted closed model and a Spark-hosted open model behind one interface. Foundry's platform-specific tooling, evaluation pipelines, agent orchestration, model routing built into the console, has no Spark equivalent, you'd stand up an alternative yourself or accept you're comparing infrastructure rather than a platform.

When Microsoft Foundry is the right choice

When a dedicated Spark is the right choice

FAQ

What is Microsoft Foundry (formerly Azure AI Foundry)?

Microsoft Foundry, called Azure AI Foundry until its latest rename, is Microsoft's platform for building and deploying AI applications on Azure, covering model deployment, evaluation, and agent tooling across a catalogue of models from OpenAI and other vendors. Microsoft has restructured and renamed this part of Azure more than once; for the specifics of any single model family it hosts, its own current documentation is the source of truth, not this page.

How is Microsoft Foundry different from Azure OpenAI?

Azure OpenAI is the OpenAI model deployment inside Microsoft's Azure platform; Foundry is the broader platform surface around it, model catalogue, evaluation tooling, and agent building blocks, that absorbed what used to be called Azure AI Studio and was renamed from Azure AI Foundry to Microsoft Foundry. For GPT-model data handling specifically, Microsoft's data, privacy and security page for Azure OpenAI is the source to check, not this page.

Can I run Foundry's models on a DGX Spark?

Closed models in Foundry's catalogue, such as OpenAI's, run only on Microsoft's infrastructure. Foundry's catalogue also lists a number of open-weight models; if a model you'd use through Foundry has open weights, you may be able to run that same model directly on a Spark instead, at flat hourly cost rather than per token. Check the specific model's licence and weight availability before assuming portability.

Who controls where my data goes on Microsoft Foundry?

Data handling depends on the specific model deployment and region you configure within Foundry, and Microsoft's own documentation for the model family in question is the governing source, not a general platform-wide guarantee. A dedicated Spark sidesteps that configuration question entirely: one machine, one region, EU-Central, no deployment options to get wrong.

Is a DGX Spark cheaper than Microsoft Foundry?

It depends on which model and deployment type you'd use in Foundry, since pricing varies by model vendor and tier, and on how continuously you can keep a Spark busy. GPUwerk did not fetch a specific current Foundry price for this page; check Microsoft's own Foundry pricing documentation for your own model. In the worked example on this page, 500 million generated tokens a month costs about $127 in Spark compute at $0.79/hour, assuming the node sustains 256 concurrent requests the entire time.

Can I switch between Microsoft Foundry and a Spark without rewriting my application?

For models Foundry serves through an OpenAI-compatible chat completions endpoint, yes, mostly: client code that only calls that endpoint shape ports with a base URL and key change. Foundry's platform-specific tooling, evaluation pipelines, agent orchestration, model routing, has no equivalent on a Spark because there's no managed platform layer, you'd build any of that yourself or run it through a gateway like LiteLLM.

GPUwerk did not fetch or verify current Microsoft Foundry pricing, regional deployment options, or catalogue-wide data-handling terms for this page, Microsoft has changed this product's shape and name more than once and its own current documentation is the source to check. Where this page's claims overlap with Azure OpenAI specifically, use Microsoft's data, privacy and security page for Azure OpenAI rather than this page as the source for that narrower product.

Related pages

First top-up: pay $20, get $30 in credit

Run your own numbers before you commit either way.

Deploy a dedicated DGX Spark and test your actual prompts against an open model before comparing quotes.

Deploy now Read the benchmarks