Skip to content

AI on AWS — Bedrock, SageMaker & the AWS AI Stack

What this is: a basic-to-solid tour of Amazon Web Services' AI portfolio — the two core platforms (Bedrock and SageMaker), the model catalog, the supporting AI services, and how it all fits together. See the cross-cloud equivalents to map these to Azure/GCP.

1. The Two Pillars

Pillar Service Use it when…
Generative AI (managed models) Amazon Bedrock You want to use foundation models via an API — chat, RAG, agents — without managing infrastructure
Custom ML (build your own) Amazon SageMaker You want to build, train, tune, and deploy your own ML/DL models end to end

Everything else on AWS is either a pre-built AI service (call an API, no training) or infrastructure these two sit on.

2. Amazon Bedrock — The GenAI Platform

A fully managed, serverless service giving one API to many foundation models. You never provision a model server; AWS runs the model, in your region, under IAM. (Deep architecture: Enterprise LLM Gateway Architecture.)

Bedrock's headline advantage is model breadth — one API for:

  • Anthropic Claude (the most-used on Bedrock)
  • Amazon Nova and Titan (AWS's own models — text, multimodal, embeddings, image)
  • Meta Llama, Mistral, Cohere, AI21, Stability AI (images)

You can swap models with a config change — no re-architecture.

Bedrock's built-in building blocks

Feature What it does
Knowledge Bases Managed RAG — connect S3/docs → auto-chunk, embed, store in a vector DB → retrieval API
Agents Multi-step tool-using agents (Bedrock AgentCore) that call APIs/Lambda to complete tasks
Guardrails Content filters, PII redaction, denied-topics, and prompt-injection defense applied to any model
Bedrock Evaluations Managed model/RAG evaluation — automatic metrics, LLM-as-a-judge, and human review (details)
Model customization Fine-tuning and continued pre-training on your data (kept private)
Flows / Prompt Management Visual orchestration and versioned prompt storage

Enterprise properties QA cares about

In-region execution, Zero Data Retention options, no-training-on-your-data terms, IAM-native access, and PrivateLink/VPC endpoints so prompts never touch the public internet.

3. Amazon SageMaker — The Custom-ML Platform

Where Bedrock is "use a model," SageMaker is "build a model." It's the end-to-end classic-ML platform (the discipline in From ML to Generative AI).

SageMaker piece Role
SageMaker Studio The IDE — notebooks, experiments, the unified workbench
Data Wrangler / Feature Store Prepare data; store and serve ML features
Training & Tuning Managed distributed training + automatic hyperparameter tuning
JumpStart Pre-trained model & solution hub (including open foundation models)
Inference endpoints Deploy models as real-time, serverless, or batch endpoints
Pipelines / Model Registry MLOps — CI/CD for models, versioning, lineage
Clarify / Model Monitor Bias detection, explainability, and production drift monitoring

Bedrock vs SageMaker in one line: Bedrock = consume managed foundation models; SageMaker = build and operate your own models. Many enterprises use both (SageMaker for tabular/predictive ML, Bedrock for GenAI).

4. Pre-Built AI Services (No Training Needed)

Call an API, get a result:

Service Does
Amazon Textract OCR / document data extraction (forms, tables)
Amazon Rekognition Image & video analysis (objects, faces, moderation)
Amazon Transcribe Speech → text
Amazon Polly Text → lifelike speech
Amazon Comprehend NLP — entities, sentiment, PII detection
Amazon Translate Machine translation
Amazon Kendra Enterprise semantic search
Amazon Q Generative AI assistant for business (Q Business) and developers (Q Developer)

Supporting infra: Amazon OpenSearch and Aurora/RDS PostgreSQL + pgvector for vector storage; S3 for the data lake; Lambda for agent tools.

5. A Typical AWS GenAI Architecture

flowchart LR
    U["App / user"] --> GW["API / gateway (IAM)"]
    GW --> BR["Amazon Bedrock<br/>(Claude / Nova)"]
    KB["S3 docs"] --> KB2["Bedrock Knowledge Base<br/>(chunk + embed)"] --> VDB["OpenSearch / pgvector"]
    BR <--> VDB
    BR --> GRD["Bedrock Guardrails"]
    BR --> EVAL["Bedrock Evaluations (CI gate)"]

6. QA / Testing Focus on AWS

  • Bedrock model calls — quality (judge/eval), latency, token cost, guardrail enforcement.
  • Knowledge Bases (RAG) — retrieval quality, temporal freshness, the RAG metrics.
  • Agents — tool-call correctness, step budgets (AI Test Strategy §3).
  • IAM & residency — RBAC on retrieval, region-boundary egress tests (AI Rollout Part 1).
  • SageMaker models — classic confusion-matrix metrics + Model Monitor drift.

Where to Go Next