Skip to content

AI Test Strategy — How to Test LLMs, RAG, Agents, MCP & Multi-Agent Systems

What this is: the umbrella strategy page. One place that answers, per technique: what are you actually testing, what are the failure modes, what layers of testing apply, and what does "done" look like? Each section links to the deep-dive guide for implementation detail.

The universal shift: traditional testing asks "does output equal X?" AI testing asks "is the output good enough, measured against a rubric, within acceptable variance — and does the system stay safe under attack?"

0. The Strategy Skeleton (Applies to Everything Below)

Every AI technique gets the same five-part treatment — only the content changes:

flowchart LR
    A["1️⃣ Deterministic layer<br/>contracts, schemas,<br/>latency, auth"] --> B["2️⃣ Quality layer<br/>golden datasets +<br/>LLM-as-judge scoring"]
    B --> C["3️⃣ Robustness layer<br/>variance, edge cases,<br/>degradation"]
    C --> D["4️⃣ Safety layer<br/>injection, jailbreaks,<br/>data leakage"]
    D --> E["5️⃣ Regression layer<br/>eval suite in CI,<br/>threshold gates, drift watch"]
Layer Question it answers Never skip because…
Deterministic Does the plumbing work? (API contract, JSON validity, latency SLA) "It's AI" doesn't exempt it from being software
Quality Are outputs correct/faithful/relevant, scored not eyeballed? Vibes-based QA doesn't survive a model upgrade
Robustness Does it hold under rephrasing, noise, load, N repeated runs? Non-determinism means one green run proves little
Safety Can it be hijacked, leaked, or made harmful? Attackers test your system even if you don't
Regression Did the prompt/model/data change make things worse? Silent quality drift is the #1 AI production bug

1. How Do You Test AI / an LLM?

System under test: a single model behind a prompt — chat, summarisation, classification, generation.

What can go wrong: hallucination, off-topic answers, format violations, unsafe content, inconsistency between runs, quality drift after a model/prompt change.

Strategy:

Layer Concretely
Deterministic Response schema (JSON validity), latency/TTFT budget, token cost per request, refusal handling
Quality Golden dataset of {input, reference/rubric}; score with LLM-as-judge on correctness, relevance, faithfulness; track hallucination rate
Robustness Run each case N times (assert distribution: mean ≥ threshold AND no run below floor); paraphrase inputs; long-context and empty-context edges
Safety Prompt injection, jailbreak suites, bias counterfactuals (swap names/genders, compare), PII leakage checks
Regression Eval suite in CI; pin the judge model; block merge if faithfulness < 0.85 or safety < 100%; re-baseline on any model upgrade 
# the shape of every LLM test: score, don't string-match
answer = call_llm(question)
score = judge(question, context, answer)          # structured JSON rubric
assert score["faithfulness"] >= 4 and score["safety"] == 5

→ Deep dives: LLM Testing Lifecycle · Evaluation Matrix · DeepEval · Cloud eval services

2. How Do You Test RAG?

System under test: a pipeline — chunking → embedding → retrieval → re-ranking → generation. Most RAG failures are retrieval failures, not LLM failures — so test every stage, not just the answer.

Strategy — test the stages, then the whole:

Stage What to assert Signature metric
Ingestion/chunking No content loss, tables intact, sane boundaries chunk completeness
Embedding Similar texts stay close; re-embedding = regression event similarity regression suite
Retrieval The right chunks are found Context Recall / Precision, Recall@k
Re-ranking Best evidence ranked first MRR / NDCG
Generation Answer grounded in retrieved chunks, cites sources Faithfulness, Answer Relevancy
End-to-end Golden Q&A built from your corpus passes pass-rate + citation accuracy

Strategy rules of thumb:

  • Build the golden set from your documents (questions whose answers you know exist) — generic benchmarks prove nothing about your corpus.
  • When an answer is wrong, bisect the pipeline: was the right chunk retrieved? If yes → generation bug; if no → retrieval/chunking bug. This one habit halves debugging time.
  • Gate CI on RAGAS-style thresholds (e.g. faithfulness ≥ 0.85) and re-run the full set on any chunking/embedding/prompt change.

→ Deep dives: RAG Automation Testing Roadmap · Ragas FAQ

3. How Do You Test an Agent?

System under test: an LLM that plans, calls tools, observes results, and loops until a task completes. You're no longer testing one answer — you're testing a trajectory.

What can go wrong: wrong tool selected, malformed arguments, hallucinated tool, infinite loops, ignoring tool errors, completing the wrong task "successfully", unsafe actions (destructive calls without confirmation).

Strategy — test at three altitudes:

Altitude What you test How
Tool-call level Correct tool chosen; arguments are valid JSON and satisfy the tool's schema; no hallucinated tools; no tool when none needed jsonschema.validate on every emitted call
Trajectory level Sensible step sequence; no loops (step budget); errors handled (retry/replan, not ignore); state carried correctly between steps trace capture + assertions on the step list
Outcome level Task actually completed; side effects correct (the record exists, the email drafted); judge scores the final result against the goal end-state verification via API/DB, LLM-as-judge on outcome 
result = run_agent("Find all overdue invoices and draft reminders")
# trajectory assertions
assert result.steps <= MAX_STEPS                       # no runaway loop
assert "query_invoices" in result.tools_used           # right tool
for call in result.tool_calls:
    validate(call.args, TOOL_SCHEMAS[call.name])       # every call schema-valid
# outcome assertions
assert db.count_draft_reminders() == db.count_overdue()  # side effect correct

Safety additions specific to agents: destructive-action gates (delete/pay/send require confirmation), scope containment (agent can't touch tools outside its manifest), and injection-via-tool-results (a malicious string in retrieved data must not hijack the plan — test it deliberately).

→ Deep dives: Evaluation Matrix (agent metrics) · Advanced API Testing §3 (tool-calling validation)

4. How Do You Test MCP?

System under test: a Model Context Protocol server — the standardised bridge between an LLM and tools/data. It's simultaneously an API (test it like one) and an AI attack surface (test it like one of those too).

Strategy — the six fronts:

Front What to test
Protocol/contract Handshake, capability negotiation, tool discovery (tools/list) returns valid schemas; every tool's input/output matches its declared JSON Schema
Tool behaviour Each tool individually: valid inputs → correct results; invalid inputs → structured errors (not crashes); timeouts honoured
Integration LLM + server together: does the model correctly use the tools the server advertises? (badly-described tools cause wrong calls — test descriptions, not just code)
Context & state Session/context propagation across calls; concurrent clients don't leak each other's state
Security Auth on every route; tool poisoning (malicious tool descriptions), injection via tool results, excessive-permission scope; rate limiting
Performance Latency per tool, behaviour under parallel tool-call bursts

Strategy insight: MCP testing is two-sided. Side A: the server obeys the protocol (pure API testing — Playwright/pytest against the endpoints). Side B: a model driving the server behaves correctly (agent testing from §3). Teams that test only side A ship servers that pass CI and still fail in real use because tool descriptions confuse the model.

→ Deep dives: MCP Testing Roadmap · MCP Servers FAQ

5. How Do You Test Multi-Agent Systems?

System under test: several agents (planner, workers, critic…) coordinating via an orchestrator. All of §3's risks, plus coordination failures — the emergent bugs that no single agent owns.

What can go wrong (beyond single-agent): hand-off drops context, two agents deadlock or ping-pong forever, wrong agent gets routed the task, one agent's hallucination becomes another's "ground truth" (error cascade), orchestration cost explodes.

Strategy — isolate, then integrate (it's the test pyramid again):

Level Analogy What you test
Unit = single agent Component test Each agent alone against its own golden tasks (§3 applies per agent)
Pair = hand-off Contract/integration test A→B hand-off: does B receive everything it needs? Schema-validate the hand-off payload like an API contract
System = full crew E2E Whole pipeline on golden scenarios: right final outcome, budget limits (steps, tokens, wall-clock) respected
Chaos = failure injection Resilience test Kill one agent mid-run, feed one agent a wrong answer — does the system detect/recover, or cascade? 
flowchart TB
    U["Unit: each agent alone<br/>golden tasks per role"] --> P["Pair: hand-off contracts<br/>schema-validated payloads"]
    P --> S["System: full pipeline E2E<br/>outcome + budgets"]
    S --> X["Chaos: inject failures<br/>assert recovery, not cascade"]

The three assertions that matter most in practice:

  1. Hand-off integrity — treat every inter-agent message as an API contract; most multi-agent bugs are "the context didn't survive the hop".
  2. Budget enforcement — max steps, max tokens, max cost per run, asserted in tests; runaway loops are a when, not an if.
  3. Error-cascade containment — deliberately give agent 1 a wrong answer and assert the critic/verifier catches it before the final output. If there's no verifier role, your test just wrote the architecture review for you.

→ Deep dives: Autonomous QA Multi-Agent Pipeline · RAG vs Agents vs Agentic RAG

6. One-Page Cheat Sheet

Technique The core question Signature test Deadliest failure mode
LLM Is the output good enough, consistently? Golden set + judge scores, N-run distribution Hallucination / silent drift
RAG Did we find the right evidence AND use it? Context recall + faithfulness, stage bisection Retrieval miss (answer exists, never found)
Agent Right tools, sane trajectory, real outcome? Schema-valid tool calls + step budget + side-effect check Confidently completing the wrong task
MCP Protocol correct AND model uses it correctly? Contract tests + model-in-the-loop tool use Passing CI, failing real use (bad tool descriptions)
Multi-agent Do hand-offs, budgets, and recovery hold? Hand-off contracts + chaos injection Error cascade / runaway loop

And for all five: safety testing (prompt injection, scope containment) is a layer, not an afterthought — and every strategy ends the same way: an eval suite in CI with explicit thresholds, because in AI systems the regression you don't measure is the one you ship.

Where to Go Next