Florian Wilde
f432122abf
Security was one section among seven in the QA Engineer. That is structurally weak: the agent that wants to recommend "production ready: YES" also judged security, and as a subsection it competes for attention with acceptance criteria, regression and test authoring. The new agent: - has no Write/Edit tools, so "never fix what you find" is enforced by the tool list rather than by instruction — it cannot launder a finding into a patch - reports PASS / FAIL / UNVERIFIED, and UNVERIFIED must reach the spec; an audit that omits what it could not check reads as all-clear - audits against .claude/rules/security.md, the same standard the developer agents are held to, so every mandatory item gets a verdict - holds a veto: BLOCK stops /deploy regardless of functional test results /qa now runs both agents in parallel and keeps the verdicts separate. The QA Engineer is explicitly forbidden from speaking for security.
4.7 KiB
name, description, argument-hint, user-invocable
| name | description | argument-hint | user-invocable |
|---|---|---|---|
| qa | Test a feature against acceptance criteria, find bugs, audit security. Use after implementation is done. | PROJ-X or feature-spec-path | true |
QA
You own the test phase for one feature. The QA Engineer sub-agent does the testing; you set up its context, then present and prioritize the results with the user.
1. Load context
- Read
features/INDEX.md— note which features are already Deployed (those are the regression surface) - Read the feature spec
git log --oneline -10— what changed recently
2. Set status
Update features/INDEX.md and the spec header to In Review. Verify by
re-reading.
3. Delegate — two agents, in parallel
Invoke both in the same turn so they run concurrently. They answer different questions and must not be merged into one agent.
QA Engineer — does the feature do what the spec promises?
- the spec path
- the list of deployed features to regression-check
- any areas the developer flagged as risky
Security Auditor — is it safe to expose to the internet?
- the spec path
- which routes, auth flows and compose services this feature touched
- whether a deployment already exists (so it can check live headers)
Wait for both reports.
The Security Auditor has a veto. If it returns BLOCK, the feature is not
Approved — regardless of how clean the functional test run was. Never average the
two verdicts into an optimistic summary, and never let the QA Engineer's
"production ready: YES" stand in for a security verdict it is not allowed to give.
4. Write results into the spec
Append this to the feature spec — not to a separate file:
---
## QA Test Results
**Getestet:** YYYY-MM-DD
**Geprüft von:** QA Engineer + Security Auditor (AI)
### Acceptance Criteria
- [x] AC-1: [Kriterium]
- [ ] AC-2: [Kriterium] — BUG-1
### Edge Cases
- [x] EC-1: [Randfall] — korrekt behandelt
### Security-Audit (Security Auditor)
**Verdikt: BLOCK | PASS-MIT-BEFUNDEN | PASS**
| Bereich | Ergebnis | Beleg |
|---------|----------|-------|
| Authentifizierung | PASS | Passkey primär, `src/lib/auth.ts:24` |
| Autorisierung | FAIL | siehe SEC-1 |
| Eingabevalidierung | PASS | Zod auf allen Write-Routen |
| Secrets | PASS | nichts in der History, `.env.example` vollständig |
| Container-Exposure | UNGEPRÜFT | Docker in dieser Umgebung nicht verfügbar |
| Security-Header | PASS | `next.config.ts:8-20` |
| Abhängigkeiten | PASS | 0 High/Critical in Prod-Deps |
| Fehler- und Log-Hygiene | PASS | |
**UNGEPRÜFT ist kein Bestanden.** Jeder solche Punkt bleibt hier stehen, bis ihn
jemand tatsächlich geprüft hat.
#### SEC-1: [Titel]
- **Severity:** Critical | High | Medium | Low
- **Ort:** `datei.ts:zeile`
- **Angriff:** was ein Angreifer konkret tut
- **Ursache:** warum es funktioniert
- **Fix gehört zu:** /backend | /frontend
### Gefundene Bugs (QA Engineer)
#### BUG-1: [Titel]
- **Severity:** Critical | High | Medium | Low
- **Reproduktion:** 1. … 2. … 3. …
- **Erwartet:** …
- **Tatsächlich:** …
### Zusammenfassung
- **Acceptance Criteria:** X von Y bestanden
- **Bugs:** N gesamt (C critical, H high, M medium, L low)
- **Security:** Verdikt + Anzahl Befunde + Anzahl UNGEPRÜFT
- **Produktionsreif:** JA / NEIN — bei Security-Verdikt BLOCK immer NEIN
Re-read the file afterwards to confirm it landed.
5. Review with the user
Present both verdicts separately — functional and security. Name every UNVERIFIED security item out loud; those are the parts nobody has checked, and burying them in the table is how they get forgotten. Then ask:
"Welche Befunde sollen zuerst behoben werden?"
6. Wrap up
- Every acceptance criterion has an individual pass/fail
- Documented and undocumented edge cases tested
- Security Auditor ran and returned a verdict
- Every UNVERIFIED security item is listed in the spec, not silently dropped
- Regression check against deployed features
- Unit tests for non-trivial logic (
npm testpasses) - E2E test per passing criterion (
npm run test:e2epasses) - QA section written into the spec and verified
features/INDEX.md→ Approved only if there are no Critical/High bugs and the security verdict is not BLOCK; otherwise stays In Review
Handoff
- Both clean → "Tests bestanden, Security-Verdikt PASS. Status Approved.
Nächster Schritt:
/deploy." - Security BLOCK → "Security-Audit blockiert: [Befunde]. Status bleibt In Review,
unabhängig von den Funktionstests. Erst
/backendbzw./frontendzum Beheben, danach nochmal/qa." - Nur Bugs → "[N] Bugs gefunden ([Aufschlüsselung]). Status bleibt In Review."
Commit
test(PROJ-X): Add QA test results for [feature name]