False Positives in Pentest Reports: Validating Before You Write

Every finding that came from a tool should be confirmed by a person before it reaches the report, in proportion to its rating, and the report should say how each one was confirmed. A false positive costs the client’s engineers time chasing nothing, and once they find one, they start doubting the findings that are real.

Why tools report weaknesses that are not there

A scanner infers. It sees a version number, a response or a timing and concludes something from it, and most of the time it is right. The cases where it is wrong fall into a few patterns worth knowing:

• Versions instead of behaviour. A check that flags a host from the version a service announces is wrong whenever the vendor fixed the flaw without changing that version, which Linux distributions do routinely.
• Responses that only look vulnerable. A delay or an error that matches an injection pattern can come from load, a slow backend or something unrelated to the input.
• Defences answering. A web application firewall or rate limiter replies in ways a tool can read as a weakness, or as a page that exists when it does not.
• The tool’s own doubt. Some tools say how sure they are. A result marked tentative is the one the tool is least sure of.

Confirm in proportion to the rating

Not every result needs the same effort. A sensible rule is that critical and high findings are reproduced by hand every time, with evidence of the effect; mediums are confirmed by hand or by a second, independent method; and lows and informational entries are checked on a sample of hosts, with the rest reported on the strength of the sample and the sample size stated.

Kind of result	How to confirm it
Missing patch, judged from a version	The installed package’s own patch level, or a credentialed check
Injection inferred from a response	Reproduce it by hand with a harmless input and show the effect
Weak transport configuration	A second tool, or the connection examined directly
Default or weak credentials	Sign in once, record that you did, and sign out
Exposed file or page	Fetch it and show what it contains, with sensitive content removed

Confirming by a different route from the one the tool used is the point. Running the same check twice tells you the tool is consistent, not that it is right. For the two tools most reports start from, turning Nessus and Burp Suite output into a client-ready report covers the steps that follow confirmation.

Say how each finding was confirmed

One line in each finding is enough: “reproduced by hand”, “confirmed from the installed package version”, or “reported by the scanner and confirmed on five of the forty hosts listed”. It tells a reviewer what to check, tells the client how much weight to give the finding, and makes the evidence section make sense. Pentest evidence that survives peer review covers the evidence itself.

When you cannot confirm it

Sometimes a result cannot be confirmed in the time, or confirming it would risk the service. Then there are two honest options. Report it as a potential finding, clearly labelled, with the reason it could not be confirmed and what the client can check themselves, rated on what the evidence does show. Or leave it out and say in the methodology that unconfirmed results were excluded. What is never honest is including it as if it were confirmed, or dropping it without a word.

The other direction: what the tools missed

Validation catches what a scanner got wrong; it does nothing about what a scanner never looks for. Authorisation flaws, business logic and findings that only matter in combination are rarely found by tools at all, which is why a report should say which parts of the scope were tested by hand. Testing authorisation properly covers the class of bug automation misses most.

Make confirmation part of review

Validation is easy to skip at the end of a long test, so make it something the reviewer checks rather than something the tester remembers. A reviewer reading a finding that came from a tool should be able to see, in the finding itself, how it was confirmed and what the evidence shows. A finding that says neither goes back to the tester with one question: how do we know this is there?

That single question catches most false positives before delivery, and it costs the reviewer seconds. It also changes how testers work, because a tester who knows the question is coming confirms as they go rather than in a hurry on the last afternoon.

Keep a record of what over-reports

The same checks produce the same false positives across clients: a version check that ignores a distribution’s backported fixes, a header check that fires on every redirect, a test that misreads one vendor’s error pages. Keep a short list of them as your team finds them, with what was wrong and how to confirm it properly, and put the note on the matching entry in your findings library. The next tester then spends a minute instead of an hour, and the firm stops rediscovering the same mistake on every engagement.

When the client says it is a false positive

Sooner or later a client will say a finding is wrong. Look at their evidence before defending yours, because sometimes they are right: the patch was applied the day before the test, or the vulnerable page is unreachable for a reason you could not see. If they are right, correct the report in a new version that says what changed and why. If they are not, show them the evidence that confirmed it. When a client disputes a finding covers the conversation.