Pentest Report QA: A Process That Survives a Busy Month

A pentest report QA process that survives a busy month makes review a step every report has to pass through, with a queue, an owner and a short checklist, rather than a favour asked of whoever has a minute. Every firm agrees reports should be reviewed before delivery, and almost every firm skips it when three engagements land in the same week, which makes the gap a design problem rather than a discipline problem.

What review is actually catching

Worth being precise about, because a reviewer who does not know what they are looking for reads the whole document and catches typography.

Class of defect	Cost if it ships	Cost to catch in review
Wrong client name or scope in the document	Severe. Instantly undermines everything else.	Seconds
Severity inconsistent with the evidence	High. Invites an argument about the score instead of the fix.	A minute per finding
Reproduction steps that do not reproduce	High. The client tries, fails, and stops trusting the finding.	Minutes, if the evidence is attached
Remediation that is not actionable	Moderate. Generates a support conversation.	A minute
Evidence containing something it should not	Severe. Another client’s hostname, a live credential, a personal record.	Seconds, if somebody looks
Coverage gaps	Moderate to severe, depending on what was missed.	A minute against the checklist
Typography and layout	Low.	Should be zero, because it should be automated

Notice the last row. If reviewers are spending their attention on formatting, the process is spending its most expensive resource on its cheapest problem.

Who should review

Not the author, obviously. Beyond that there are three workable models and one that does not work.

• Peer review. Another tester at the same level. Best at technical accuracy, weakest at house consistency, and the only model that scales in a small firm.
• Senior review. A lead reviews everything. Strongest on consistency and client fit, and the first thing to break when the lead is busy, which is always.
• Split review. A peer checks the technical content, a lead checks the summary and the client-facing framing. More overhead, and the best quality per hour spent once you are past three or four testers.
• Rotating "whoever is free". The model that does not work. Nobody develops a reviewer’s eye, and standards drift because there are no standards, only preferences.

The checklist a reviewer should actually use

1. The obvious. Right client, right scope, right dates, right version of the template.
2. Every finding against its evidence. Does the severity follow? Do the steps reproduce? Is the affected asset list complete? This is where evidence that survives peer review pays for itself.
3. Evidence hygiene. Anything in a screenshot that should not leave your building, including other clients.
4. Coverage. The checklist for each phase, complete or with a stated reason.
5. The summary against the findings. The counts must match, and the themes must be visible in the detail.
6. The client fit. Read the first page as the recipient. Does it tell them what to do on Monday?

A reviewer who reads the report from the front is reading a document. A reviewer who reads each finding against its evidence is reviewing.

Making it survive the bad week

• Give review a queue. Work sits in it, visibly, until somebody takes it. Invisible work is skippable work.
• Time-box it. Twenty minutes per assessment is a sensible target for a competent reviewer with the evidence to hand. If it is taking two hours, the problem is upstream.
• Comment in place. On the finding, not in an email. Feedback that has to be reconciled from a thread costs more than it saves.
• Separate release from save. If nothing reaches a client until somebody releases it, skipping review requires a decision rather than an oversight.
• Feed the library. A reviewer who rewrites a paragraph has produced the better version. Capture it in a findings library your team actually uses, or you will pay for it again next quarter.

When the reviewer and the author disagree

They will, and a process that has no answer for it stalls on exactly the reports that matter most. The workable rule is a split of ownership. The author owns the facts: what was observed, what was exploited, what the evidence shows. The reviewer owns the house standard: how a weakness is worded, how it is scored, what a recommendation must contain.

So a reviewer applies wording and consistency fixes directly, and sends anything that changes a fact, a severity or a recommendation back to the author with the reason. Where the two still disagree on a severity, the lead decides, and the decision becomes part of the written house position rather than a one-off. Two testers arguing the same scoring question twice is a sign the position was never written down.

Keep the comments after the report ships. A quarter of review comments is the most honest training material a firm owns, because it shows exactly where its writers and its standard part company.

The measures worth watching

• What proportion of assessments were reviewed before release last month? Anything under a hundred per cent is a process gap, not a people gap.
• How long does a review take? Rising times mean the upstream work is getting worse.
• What is being caught? If it is mostly formatting, automate it. If it is mostly severity, your scoring needs a house position, and the usual CVSS scoring mistakes are the place to start writing it.
• What reached a client and came back? Every one of those is a review criterion you did not have.

The firms with the best reputation for report quality are rarely the ones with the best writers. They are the ones where nothing reaches a client without a second person having read it against the evidence.