Skip to content

The Learning Path — Manual QA to AI QA, Stage by Stage

How to use this library. The 40+ guides here are organised into a deliberate curriculum. Follow the stages in order — each one builds on the last, states what you'll be able to do afterwards, and ends with a checkpoint you can test yourself against. Skip a stage only if you genuinely pass its checkpoint already.

Pace: at ~5 hours/week this is a 10–14 week transformation. Faster with prior automation experience.

The Journey at a Glance

flowchart LR
    S0["0 · Orient<br/>why & what"] --> S1["1 · Foundations<br/>AI basics + a language"]
    S1 --> S2["2 · Automation<br/>Playwright + API"]
    S2 --> S3["3 · Data<br/>pipelines + quality"]
    S3 --> S4["4 · AI Testing<br/>LLM · RAG · agents"]
    S4 --> S5["5 · Platforms<br/>cloud AI + gateway"]
    S5 --> S6["6 · Operate & Lead<br/>observability · strategy · security"]

Manual-tester head start: your existing skills — test design, risk thinking, exploratory instinct, defect writing — transfer completely. This path adds tooling and AI knowledge on top of them; it does not replace them.

Stage 0 · Orient (half a day)

Goal: understand why QA is changing and what the destination roles look like.

Read Why
Why QA Is Changing The motivation — what "testing intelligence" means
AI QA Role Requirements The target — what employers actually ask for
QA → AI QA — 6-Week Transition The condensed plan this path expands

✅ Checkpoint: you can explain, in two minutes, why deterministic pass/fail testing is insufficient for AI systems — and name the role you're aiming for.

Stage 1 · Foundations (1–2 weeks)

Goal: speak AI fluently and read/write basic test code.

Read Why
AI Fundamentals for Beginners ML → neural nets → LLMs → RAG, from zero
From ML to Generative AI Why the industry moved — with fraud/spam case studies
TypeScript Cheat Sheet The language of modern UI automation
Python for Data Testing The language of data & AI testing (pandas, DataFrames)

✅ Checkpoint: you can explain LLM vs RAG to a colleague; you can read a Playwright test and a pandas script without getting lost.

Stage 2 · Test Automation (2–3 weeks)

Goal: go from manual execution to building and running automated suites — the biggest single jump.

Read in order (each builds on the previous):

  1. Playwright Project Anatomy — understand every file before writing tests
  2. Playwright Framework Tutorial — Page Objects, fixtures, data, CI
  3. API Testing with Playwright — the fast layer under the UI
  4. Playwright Advanced Guide — isolation, parallelism, reporting, CI/CD
  5. Advanced API Testing — WebSockets, media, rate limits (senior depth)

✅ Checkpoint: you can build a small framework from scratch — Page Objects + API setup + one CI workflow — and explain why each file exists.

Stage 3 · Data Testing (1 week)

Goal: test the data that feeds every AI system — most AI failures start as data failures.

Read Why
Data Testing — Pipelines & Data Quality Pipelines, ingestion, snapshots, upstream/downstream, structured vs unstructured
Python for Data Testing (revisit §4) Run the pytest data-quality suite for real

✅ Checkpoint: given a source table and a target table, you can write reconciliation, uniqueness, referential-integrity, and freshness tests in pandas.

Stage 4 · AI Testing (3–4 weeks — the core)

Goal: evaluate LLMs, RAG pipelines, and agents like a professional.

4a — Understand the systems: RAG vs Agents vs Agentic RAG · MCP Servers FAQ

4b — Learn the strategy, then the metrics:

  1. AI Test Strategy — the umbrella: how to test LLM / RAG / agents / MCP / multi-agent
  2. LLM Testing Lifecycle — the process view
  3. LLM & Agent Evaluation Matrix — which metric answers which question
  4. RAG Automation Testing Roadmap — stage-by-stage RAG testing
  5. RAG Evaluation Metrics — Deep Dive — the mathematics (interview depth)

4c — Learn the tools: Ragas FAQ · DeepEval FAQ · Deep Dives — Confusion Matrix · SageMaker · Agents · Telemetry

4d — Let AI help you test: AI QA Agents Catalogue · Test-Case Generator Agent · Autonomous QA Multi-Agent Pipeline

✅ Checkpoint: you can design a golden dataset, run RAGAS/DeepEval against a RAG system, gate a CI pipeline on faithfulness ≥ 0.85, and explain the difference between a retrieval failure and a generation failure.

Stage 5 · Platforms & Architecture (1 week)

Goal: know where AI systems run in the enterprise and how the pieces connect.

Read Why
Cloud AI Platforms — Overview The AWS/Azure/GCP map + equivalents table
One deep page for your cloud: AWS / Azure / GCP Depth where you work
Enterprise LLM Gateway Architecture Proxy → Bedrock → Claude, with Postgres — the standard enterprise design
Cloud LLM Evaluation Tools Managed evaluation as your release gate

✅ Checkpoint: you can whiteboard the enterprise gateway architecture and name each cloud's GenAI platform, ML platform, and evaluation service.

Stage 6 · Operate, Secure & Lead (2 weeks)

Goal: the senior layer — production observability, security, and QA-leadership strategy.

Read Why
LLM Observability — LiteLLM & Langfuse Track usage, capture logs, see the problems
Prompt Injection — Complete Guide + Red/Blue/Purple Teams The adversarial dimension
AI Rollout Strategy — 3-Part Blueprint Governance, risk, and sign-off — the QA-manager view
Commercial LLM / MCP Testing Tools The market landscape

✅ Checkpoint: you can present an AI quality strategy to leadership — risks, test layers, metrics, gates, observability — and defend it. That is the AI QA Lead interview.

After the Path