Deduplicating Vulnerability Scanner Findings Before They Reach a Developer

Run two or three scanners against the same estate and the same weakness comes back several times: under each tool’s own name, split by host and by port, sometimes rolled into a family and sometimes listed bulletin by bulletin. Deduplicating scanner findings means turning that into one finding per fix, grouped by the issue rather than by the title a tool happened to give it, with every host listed and each host’s evidence kept. Get it wrong one way and a developer is handed the same fix four times; get it wrong the other way and a critical vulnerability disappears inside a routine patching item.

Why the same issue arrives several times

Scanners are built to be complete, not to agree with each other. Each has its own catalogue of checks, its own identifier for every check and its own wording for the title, so the same weak cipher suites on the same service come back from three tools as three results with three names. Even within one tool, results are per host and often per port: one misconfigured template deployed to forty servers is forty results.

Granularity differs too. One scanner reports a missing operating system update as a single rollup; another lists each bulletin it covers as a result of its own. A TLS scanner reports each deprecated protocol separately, where a vulnerability scanner may report them together. None of this is wrong from the scanner’s point of view. It is just not the shape a report should have, because the client fixes issues, not scanner results.

What counts as the same issue

The useful test is the fix. Two results are the same issue when one remediation closes both: disable the same protocol, apply the same configuration, upgrade the same component. That test is easy to state and hard to apply by title alone, so matching works best on the strongest signal available, in order:

1. The scanner’s own identifier for the check. A plugin ID, a QID, a template ID or a check name is stable across scans and across wording changes, and it can be mapped once to the issue it represents.
2. The CVEs a result names. Two results naming the same advisories are, for the purposes of remediation, the same issue, whatever each tool called it.
3. The title, normalised. Case, punctuation and trailing qualifiers such as a bracketed note on how the check was run should not split a group.
4. Patterns and families. A pattern over the title catches the variants a list of exact titles misses, and a scanner’s own family, used carefully, turns a long list of missing bulletins into the one patching finding it really is.

Each step is less certain than the one before it, which is why the order matters: an identifier match should never be overridden by a looser title match.

Merging too much

The expensive mistake is the merge that hides something. The classic case is a named, actively exploited vulnerability folded into a generic “missing security updates” finding because the scanner files both under the same family. The rollup is medium or high and gets scheduled for the next patch window; the named vulnerability needed fixing this week. Specific issues should be matched before families are, so MS17-010 on a host stays MS17-010 even though the same host also has a dozen routine updates outstanding.

The second is merging away context. One issue on an internet-facing host and on an internal one is still one finding, but the finding must keep the worst rating, the score and vector it came from, and each host’s own evidence, so a reviewer can see that the exposure is not uniform. A merged finding that keeps only the first host’s output is a finding nobody can verify.

Merging too little

The opposite mistake is quieter but just as costly. Title variants that differ only in punctuation or a qualifier, one tool naming the CVE while another names the vendor bulletin, a TLS scanner’s check name against a vulnerability scanner’s plugin title: each leaves the client with two findings for one fix. Engineers notice. They close one ticket, find the second still open, and start to treat the report as noise, which is the opposite of what a report is for.

It also distorts everything counted afterwards. Severity totals, the number of issues fixed between tests and the time fixes take are all wrong when one issue is counted three times.

What the merged finding should carry

• One title, the firm’s name for the issue rather than whichever scanner’s came first, and one description and remediation, written once.
• Every affected host any of the tools reported, with each host’s output kept as evidence under a heading naming the host and port.
• The worst severity of the group, and the highest score with the vector it came from, so the metrics can be checked against this client.
• A line saying what it stands for: which tools reported it, under which titles, on how many hosts. A reviewer who disagrees with a grouping can see it at once.
• The references gathered from all of them, without duplicates.

Where the firm has its own write-up of the issue, that wording should win over any scanner’s; scanner prose is written for the scanner’s users, not for the client’s engineers. Writing remediation a developer will action covers what that wording should say.

Why it matters after the report

Deduplication pays off most after delivery. One finding per issue is one ticket per fix in the client’s tracker, one row to retest, and one line in any measure of how quickly the client fixes things. A later scan should add hosts to the findings already there rather than duplicate them, and a retest should be able to say of each issue, once, whether it is fixed. The same discipline is what makes it possible to track a client’s exposure from one engagement to the next, which is the subject of measuring what clients fix between pentests.

It is also the honest version of the report. A client paying for testing is paying for judgement, and deciding that twenty-nine results are really two issues is part of it. Vulnerability scan or penetration test explains the difference to a client who has only ever seen raw scanner output.