Template
Florian Wilde
f432122abf
Security was one section among seven in the QA Engineer. That is structurally weak: the agent that wants to recommend "production ready: YES" also judged security, and as a subsection it competes for attention with acceptance criteria, regression and test authoring. The new agent: - has no Write/Edit tools, so "never fix what you find" is enforced by the tool list rather than by instruction — it cannot launder a finding into a patch - reports PASS / FAIL / UNVERIFIED, and UNVERIFIED must reach the spec; an audit that omits what it could not check reads as all-clear - audits against .claude/rules/security.md, the same standard the developer agents are held to, so every mandatory item gets a verdict - holds a veto: BLOCK stops /deploy regardless of functional test results /qa now runs both agents in parallel and keeps the verdicts separate. The QA Engineer is explicitly forbidden from speaking for security.
143 lines
4.7 KiB
Markdown
143 lines
4.7 KiB
Markdown
---
|
|
name: qa
|
|
description: Test a feature against acceptance criteria, find bugs, audit security. Use after implementation is done.
|
|
argument-hint: "PROJ-X or feature-spec-path"
|
|
user-invocable: true
|
|
---
|
|
|
|
# QA
|
|
|
|
You own the test phase for one feature. The **QA Engineer** sub-agent does the
|
|
testing; you set up its context, then present and prioritize the results with the
|
|
user.
|
|
|
|
## 1. Load context
|
|
|
|
1. Read `features/INDEX.md` — note which features are already **Deployed**
|
|
(those are the regression surface)
|
|
2. Read the feature spec
|
|
3. `git log --oneline -10` — what changed recently
|
|
|
|
## 2. Set status
|
|
|
|
Update `features/INDEX.md` and the spec header to **In Review**. Verify by
|
|
re-reading.
|
|
|
|
## 3. Delegate — two agents, in parallel
|
|
|
|
Invoke both in the same turn so they run concurrently. They answer different
|
|
questions and must not be merged into one agent.
|
|
|
|
**QA Engineer** — does the feature do what the spec promises?
|
|
- the spec path
|
|
- the list of deployed features to regression-check
|
|
- any areas the developer flagged as risky
|
|
|
|
**Security Auditor** — is it safe to expose to the internet?
|
|
- the spec path
|
|
- which routes, auth flows and compose services this feature touched
|
|
- whether a deployment already exists (so it can check live headers)
|
|
|
|
Wait for both reports.
|
|
|
|
The Security Auditor has a **veto**. If it returns `BLOCK`, the feature is not
|
|
Approved — regardless of how clean the functional test run was. Never average the
|
|
two verdicts into an optimistic summary, and never let the QA Engineer's
|
|
"production ready: YES" stand in for a security verdict it is not allowed to give.
|
|
|
|
## 4. Write results into the spec
|
|
|
|
Append this to the feature spec — not to a separate file:
|
|
|
|
```markdown
|
|
---
|
|
|
|
## QA Test Results
|
|
|
|
**Getestet:** YYYY-MM-DD
|
|
**Geprüft von:** QA Engineer + Security Auditor (AI)
|
|
|
|
### Acceptance Criteria
|
|
- [x] AC-1: [Kriterium]
|
|
- [ ] AC-2: [Kriterium] — BUG-1
|
|
|
|
### Edge Cases
|
|
- [x] EC-1: [Randfall] — korrekt behandelt
|
|
|
|
### Security-Audit (Security Auditor)
|
|
|
|
**Verdikt: BLOCK | PASS-MIT-BEFUNDEN | PASS**
|
|
|
|
| Bereich | Ergebnis | Beleg |
|
|
|---------|----------|-------|
|
|
| Authentifizierung | PASS | Passkey primär, `src/lib/auth.ts:24` |
|
|
| Autorisierung | FAIL | siehe SEC-1 |
|
|
| Eingabevalidierung | PASS | Zod auf allen Write-Routen |
|
|
| Secrets | PASS | nichts in der History, `.env.example` vollständig |
|
|
| Container-Exposure | UNGEPRÜFT | Docker in dieser Umgebung nicht verfügbar |
|
|
| Security-Header | PASS | `next.config.ts:8-20` |
|
|
| Abhängigkeiten | PASS | 0 High/Critical in Prod-Deps |
|
|
| Fehler- und Log-Hygiene | PASS | |
|
|
|
|
**UNGEPRÜFT ist kein Bestanden.** Jeder solche Punkt bleibt hier stehen, bis ihn
|
|
jemand tatsächlich geprüft hat.
|
|
|
|
#### SEC-1: [Titel]
|
|
- **Severity:** Critical | High | Medium | Low
|
|
- **Ort:** `datei.ts:zeile`
|
|
- **Angriff:** was ein Angreifer konkret tut
|
|
- **Ursache:** warum es funktioniert
|
|
- **Fix gehört zu:** /backend | /frontend
|
|
|
|
### Gefundene Bugs (QA Engineer)
|
|
|
|
#### BUG-1: [Titel]
|
|
- **Severity:** Critical | High | Medium | Low
|
|
- **Reproduktion:** 1. … 2. … 3. …
|
|
- **Erwartet:** …
|
|
- **Tatsächlich:** …
|
|
|
|
### Zusammenfassung
|
|
- **Acceptance Criteria:** X von Y bestanden
|
|
- **Bugs:** N gesamt (C critical, H high, M medium, L low)
|
|
- **Security:** Verdikt + Anzahl Befunde + Anzahl UNGEPRÜFT
|
|
- **Produktionsreif:** JA / NEIN — bei Security-Verdikt BLOCK immer NEIN
|
|
```
|
|
|
|
Re-read the file afterwards to confirm it landed.
|
|
|
|
## 5. Review with the user
|
|
|
|
Present both verdicts separately — functional and security. Name every UNVERIFIED
|
|
security item out loud; those are the parts nobody has checked, and burying them in
|
|
the table is how they get forgotten. Then ask:
|
|
> "Welche Befunde sollen zuerst behoben werden?"
|
|
|
|
## 6. Wrap up
|
|
|
|
- [ ] Every acceptance criterion has an individual pass/fail
|
|
- [ ] Documented and undocumented edge cases tested
|
|
- [ ] Security Auditor ran and returned a verdict
|
|
- [ ] Every UNVERIFIED security item is listed in the spec, not silently dropped
|
|
- [ ] Regression check against deployed features
|
|
- [ ] Unit tests for non-trivial logic (`npm test` passes)
|
|
- [ ] E2E test per passing criterion (`npm run test:e2e` passes)
|
|
- [ ] QA section written into the spec and verified
|
|
- [ ] `features/INDEX.md` → **Approved** only if there are no Critical/High bugs
|
|
**and** the security verdict is not BLOCK; otherwise stays **In Review**
|
|
|
|
## Handoff
|
|
|
|
- Both clean → "Tests bestanden, Security-Verdikt PASS. Status **Approved**.
|
|
Nächster Schritt: `/deploy`."
|
|
- Security BLOCK → "Security-Audit blockiert: [Befunde]. Status bleibt **In Review**,
|
|
unabhängig von den Funktionstests. Erst `/backend` bzw. `/frontend` zum Beheben,
|
|
danach nochmal `/qa`."
|
|
- Nur Bugs → "[N] Bugs gefunden ([Aufschlüsselung]). Status bleibt **In Review**."
|
|
|
|
## Commit
|
|
|
|
```
|
|
test(PROJ-X): Add QA test results for [feature name]
|
|
```
|