Skip to content

AI on Azure — AI Foundry, Azure OpenAI & Azure AI Services

What this is: a basic-to-solid tour of Microsoft Azure's AI portfolio — the GenAI platform (AI Foundry + Azure OpenAI), the custom-ML platform (Azure Machine Learning), and the pre-built AI Services shelf. See the cross-cloud equivalents to map these to AWS/GCP.

1. The Two Pillars

Pillar Service Use it when…
Generative AI (managed models) Azure AI Foundry (+ Azure OpenAI Service) You want to build GenAI apps/agents on foundation models with enterprise governance
Custom ML (build your own) Azure Machine Learning You want to build, train, and deploy your own ML/DL models end to end

Azure's distinctive edge: exclusive access to OpenAI's GPT-4o / o-series models, deep Entra ID (identity) integration, and tight ties to Microsoft 365 / GitHub.

2. Azure AI Foundry — The GenAI Platform

Azure AI Foundry (the evolution of Azure AI Studio) is the unified platform to build, evaluate, and deploy generative AI apps and agents. It wraps several pieces:

Foundry piece Role
Azure OpenAI Service Hosted OpenAI models — GPT-4o, GPT-4.1, o3, o4-mini, embeddings, DALL·E — with enterprise SLAs, in-region hosting, and no-training terms
Model Catalog 1,800+ models beyond OpenAI — Llama, Mistral, Cohere, and open models — deployable as managed or serverless endpoints
Agent Service Build and orchestrate tool-using agents
Prompt Flow Visual authoring, chaining, and evaluation of prompt pipelines
Evaluation Built-in evaluators — groundedness, relevance, coherence, plus risk & safety (violence, self-harm, jailbreak) — pre-prod and in production (details)
Content Safety Filters for hate/violence/self-harm, prompt-shield (injection defense), protected-material detection

Enterprise properties QA cares about

Entra ID (Azure AD) identity, private networking (Private Link / VNet), in-region data residency, customer-managed keys, and content-safety governance evidence for regulated sign-off.

3. Azure Machine Learning — The Custom-ML Platform

The end-to-end classic-ML platform (equivalent to AWS SageMaker / Vertex training):

AML piece Role
Studio / Notebooks The workbench IDE
Designer Low-code drag-drop ML pipelines
Automated ML (AutoML) Auto model selection + hyperparameter tuning
Compute & Training Managed clusters for distributed training
Endpoints Real-time and batch model deployment
MLOps Pipelines, model registry, versioning, CI/CD
Responsible AI dashboard Fairness, explainability, error analysis

Foundry vs Azure ML: Foundry = generative AI apps on managed foundation models; Azure ML = build/operate your own models. They interoperate (Foundry can consume AML-deployed models).

4. Azure AI Services (Pre-Built — formerly "Cognitive Services")

Call an API, no training:

Service Does
Azure AI Document Intelligence OCR / structured extraction from forms & docs
Azure AI Vision Image analysis, OCR, spatial analysis
Azure AI Speech Speech-to-text, text-to-speech, translation
Azure AI Language NLP — entities, sentiment, PII, summarization, Q&A
Azure AI Translator Machine translation
Azure AI Search Enterprise search + vector search (the RAG retrieval layer)
Azure AI Content Safety Standalone moderation for text & images

Assistants: Microsoft 365 Copilot (productivity) and Copilot Studio (build custom copilots/agents low-code).

5. A Typical Azure GenAI Architecture

flowchart LR
    U["App / user"] --> ENTRA["Entra ID auth"]
    ENTRA --> AOAI["Azure OpenAI<br/>(GPT-4o / o-series)"]
    DOCS["Blob docs"] --> SRCH["Azure AI Search<br/>(chunk + vector index)"]
    AOAI <--> SRCH
    AOAI --> CS["AI Content Safety<br/>(prompt shield + filters)"]
    AOAI --> EVAL["Foundry Evaluation (CI gate)"]

This is the classic "Azure OpenAI + Azure AI Search" RAG pattern — the most common enterprise design on Azure.

6. QA / Testing Focus on Azure

  • Azure OpenAI calls — quality (Foundry evaluators), latency, token cost, content-safety enforcement.
  • AI Search RAG — retrieval quality, hybrid (keyword+vector) tuning, RAG metrics.
  • Agents / Copilot Studio — tool-call correctness, task adherence (AI Test Strategy).
  • Entra ID & residency — RBAC on retrieval, region-boundary tests (AI Rollout Part 1).
  • Safety — jailbreak/prompt-shield evaluators; prompt injection guide.

Where to Go Next