Test Cases Generator AI Agent — Architecture Reference¶
Stack: JIRA MCP + RAG + LangGraph
Pattern: Intent-driven Multi-Agent Workflow
Output: Structured Test Case Pack with traceability
Overview¶
An end-to-end agentic system that accepts a natural-language QA query (e.g. "Generate only negative test cases for SCRUM-5"), retrieves context from both JIRA and a RAG knowledge base, routes to specialist test-generation agents, and returns a professional test case pack.
Architecture Flow¶
User Query
│
▼
Streamlit UI
│
▼
Input Guardrail ← Validates request is QA/JIRA-safe
│
▼
Reformulator ← Converts query → structured JSON
│ { jira_key, intent, test_types, output_format }
▼
Orchestrator ← Decides which subgraph paths to run
│
├──────────────────────────────────────────────┐
▼ ▼
JIRA Subgraph RAG Subgraph
│ │
▼ ▼
Agentic Guardrail Agentic Guardrail
(safe read-only JIRA access) (safe RAG access)
│ │
▼ ▼
JIRA MCP Client ──────────────┐ Retrieval from ChromaDB
│ Fallback: │ ├─ testing_best_practices.md
▼ Direct REST │ └─ brd_sample.md
Custom JIRA MCP Server │ │
│ API if MCP │ ▼
▼ unavailable ─┘ Search Evaluation
JIRA Cloud API (checks relevance)
│ │
▼ ▼
Search Evaluation Answer Evaluation
(story quality & completeness) (summarises testing guidance)
│ │
▼ ▼
Output: JIRA story + Output: RAG context +
Acceptance Criteria testing guidance
│ │
└──────────────┬───────────────────────────────┘
▼
Aggregator
Combines: structured query + JIRA story + RAG guidance
│
▼
Test Type Router
Invokes only requested specialist agents
│
┌──────┬───────┼────────┬──────────┐
▼ ▼ ▼ ▼ ▼
Functional Negative Edge Case API Test Regression
Agent Agent Agent Agent Agent
│
▼
Output Guardrail
Removes unsafe or unsupported content
│
▼
Tone Check
Improves clarity and professional QA style
│
▼
Final Test Case Pack
Component Breakdown¶
Entry & Validation Layer¶
| Component | Role |
|---|---|
| Streamlit UI | Chat-style front-end for QA engineer queries |
| Input Guardrail | Validates request is QA/JIRA-safe before processing |
| Reformulator | Parses natural language → structured JSON: { jira_key, intent, test_types, output_format } |
| Orchestrator | Routing brain — decides whether to invoke JIRA subgraph, RAG subgraph, or both |
JIRA Subgraph¶
Fetches the live story and acceptance criteria from JIRA Cloud.
| Component | Role |
|---|---|
| Agentic Guardrail | Enforces read-only JIRA access — no mutations |
| JIRA MCP Client | Primary path: connects via custom MCP server |
| Custom JIRA MCP Server | Wraps JIRA Cloud API behind an MCP interface |
| JIRA Cloud API | Fallback: direct REST call if MCP is unavailable |
| Search Evaluation | Scores story quality and completeness before passing downstream |
| Output | JIRA story text + acceptance criteria |
Key design note: MCP-first with REST fallback — the system degrades gracefully when the MCP server is unavailable.
RAG Subgraph¶
Retrieves testing best practices and BRD context from a local vector store.
| Component | Role |
|---|---|
| Agentic Guardrail | Validates safe RAG access |
| ChromaDB Retrieval | Vector similarity search over knowledge documents |
| Knowledge Documents | testing_best_practices.md, brd_sample.md |
| Search Evaluation | Checks retrieval relevance — filters low-quality chunks |
| Answer Evaluation | Summarises retrieved context into actionable testing guidance |
| Output | RAG context + testing guidance summary |
Aggregation & Routing Layer¶
| Component | Role |
|---|---|
| Aggregator | Merges: structured query + JIRA story + RAG guidance into a unified prompt context |
| Test Type Router | Reads intent and test_types from reformulated JSON — invokes only the relevant specialist agents |
Specialist Test Generation Agents¶
Five independent agents, each focused on one test category:
| Agent | Generates |
|---|---|
| Functional Test Agent | Happy-path, core behaviour tests |
| Negative Test Agent | Invalid inputs, error handling, boundary violations |
| Edge Case Agent | Boundary values, unusual but valid scenarios |
| API Test Agent | Contract tests, HTTP status codes, payload validation |
| Regression Agent | Tests ensuring existing functionality isn't broken |
Only the agent(s) matching the user's intent are invoked — not all five every time.
Output Layer¶
| Component | Role |
|---|---|
| Output Guardrail | Removes unsafe, unsupported, or hallucinated content from generated test cases |
| Tone Check | Rewrites for professional QA style and clarity |
Final Test Case Pack¶
The deliverable includes:
- Story Summary — plain-English restatement of the JIRA story
- Assumptions — what the agent assumed when gaps existed
- Requested Test Cases — structured test steps in the requested format
- Traceability Matrix — maps each test case back to an acceptance criterion
- Coverage Summary — states what is and isn't covered
Key Architectural Patterns¶
Guardrails at Multiple Levels¶
Three independent guardrail checkpoints: 1. Input Guardrail — validates the user query before any work begins 2. Agentic Guardrails (×2) — one per subgraph, enforcing safe data access 3. Output Guardrail — sanitises generated content before delivery
Intent-Driven Routing¶
The Reformulator extracts intent and test_types upfront. The Test Type Router uses these fields to invoke only the relevant specialist agents — avoiding unnecessary LLM calls and keeping outputs focused.
MCP + REST Fallback¶
JIRA access is MCP-first (structured, auditable) with a silent REST API fallback. This pattern is reusable for any external tool integration in agentic systems.
Dual-Source Context¶
Tests are grounded in both live story data (JIRA) and knowledge-base guidance (RAG). Neither source alone is sufficient — JIRA gives specifics, RAG gives testing standards.
Technology Stack¶
| Layer | Technology |
|---|---|
| Orchestration | LangGraph (state-machine multi-agent) |
| UI | Streamlit |
| Vector Store | ChromaDB |
| JIRA Integration | Custom MCP Server + JIRA Cloud REST API |
| Agent Framework | LangGraph nodes (one per specialist agent) |
| LLM | Not specified — drop-in (OpenAI / local / Bedrock) |
Interview Sound-Bites¶
"We use three independent guardrail checkpoints — one at input, one per data-access subgraph, and one at output. Each guards a different attack surface."
"The Reformulator is the key to intent-driven routing. It converts natural language into a structured JSON contract that every downstream component reads from."
"MCP-first with REST fallback means the system is both auditable under normal conditions and resilient when infrastructure degrades."
"Specialist agents are invoked selectively — if the user asks for negative tests only, the Functional, Edge Case, API, and Regression agents never run. That's cost-efficient and output-focused."
"The traceability matrix is a first-class output. Every generated test case traces back to an acceptance criterion — which is what regulated environments require."