Reference pattern
Embedded Test-Automation Orchestration Layer
This reference pattern illustrates one way to give AI-assisted test automation consistent, repository-aware context instead of starting from zero each session.
← Back to In PracticeProblem
Generic coding assistants can generate tests, but they often lack the repository-specific context needed to produce consistent, maintainable automation.
Conceptual architecture
Place a lightweight orchestration layer inside the automation repository. Route work to narrow specialist capabilities, maintain reusable project context and rules, and apply deterministic quality gates after changes.
Transferable principles
- Route requests through explicit routing rules, and ask for clarification when the intent is ambiguous.
- Diagnose a failing test before repairing it: the cause may be a product defect, a problem in the test, or unclear intended behavior.
- Keep shared repository conventions in one place that every capability uses.
- Run quality gates as automated checks after every change. A passing check is a signal for review, not proof that the change is correct.
- Match review depth to risk, and confirm a fix preserves meaningful assertions and intended behavior — never weaken an assertion just to make a check pass.
- Record the validated cause and lesson in shared context, so the same mistake becomes less likely next time.
Engineer Request
Repo-Aware Orchestrator
Classifies before doing any work
Routing Decision
Explicit rules; clarify if ambiguous
Specialist Capability
- Create test
- Diagnose failure
- Review / refactor
Repository Rules + Context
Shared conventions and project knowledge
Quality Gates
Automated checks before review
Human Review
Depth suited to risk
Connection to Hyper-Agile Quality Engineering
Shared conventions and project knowledge reach everyone working in the repository, with context and checks appropriate to each change. Diagnosis comes before repair, review depth follows risk, and the engineer confirms a fix keeps meaningful assertions — a passing check informs that decision rather than replacing it. Validated causes are recorded, so the next change starts better informed.
Framework pillars
- Risk-Based Validation Depth
- Continuous Quality Signals
- Enabled Ownership
- Informed Confidence
Supporting practices: Review-first AI, Reusable quality context
Hypothetical example
An engineer on a fictional travel-booking product asks for help with a failing “change seat” test.
- 1.The orchestration layer classifies the request and routes it to diagnosis before anything is treated as a test defect.
- 2.Diagnosis compares the failure with the intended behavior. A product defect would be reported rather than fixed in the test; unclear intended behavior would be clarified with the relevant Product, Engineering, or QE contributors.
- 3.Here the cause is in the test: a timing assumption no longer holds. The capability proposes a fix that follows the repository's conventions for waiting and locating elements.
- 4.Automated checks run, and the engineer reviews the fix — ordinary maintenance, so a single review is enough — confirming it keeps meaningful assertions and intended behavior rather than merely making the check pass.
- 5.The validated cause and lesson are recorded in shared context, so similar tests avoid the same problem.
This is a conceptual reference pattern. Examples are hypothetical, and any implementation should be adapted to each organization's tools, risks, and constraints.
The architecture is the visible part
Deciding what deserves deeper validation, which signals can be trusted, and who owns the decision is the operating model behind this pattern. An organizational engagement can start with one workflow like this one; workshops can help a team apply the approach. The operating model is developed in Hyper-Agile Testing.
Other reference patterns