Stopping AI From Inventing Findings in Pentest Reports

To stop an AI inventing findings in a pentest report, give it nothing to invent from: draft each finding only from the evidence the tester captured, keep every value in that evidence exactly as written, and never ask the model for facts it was not given, such as a CVE number or a version. Then treat what it writes as a draft that a person keeps, reworks or discards, check every number and hostname against the evidence, and review the report against a checklist before it reaches the client.

Why models invent things in a finding

A language model writes the most plausible words for the gap in front of it. A finding is a set of slots (version, CVE, hosts, impact, severity, remediation), and a slot the evidence leaves empty reads to the model as a request to fill it. It has read thousands of advisories, so what it fills in sounds right, in your house style.

• A version. The evidence shows a path traversal on an Apache server whose banner hides its version. A model that has read about CVE-2021-41773 writes that the server runs 2.4.49.
• A CVE. An outdated library collects every CVE the model associates with its name, including ones fixed in releases older than the one you found.
• A host. One affected host becomes "several hosts across the estate", or a hostname from a library entry written for another client joins the affected assets.
• An impact. One record read from a test account becomes "access to all customer data".
• A severity. The draft scores the title, so "SQL injection" comes out critical whether or not the injection point reached anything.

Two more sources are easy to miss: a prompt that asks for "a complete finding with CVE references and affected versions", and the tester's own notes. Asked to turn "maybe the v2 API too?" into prose, a model writes "the v2 API is also affected", and nobody invented that on purpose.

Draft only from what the tester captured

The first control is the input. Draft a client's finding from that client's evidence: the notes, the commands and their output, the requests and responses, the screenshots, pasted as rough as they were taken. A finding drafted from its title, or from the model's memory of the weakness, has nothing to be faithful to. The better the capture, the less there is to fill, which is one more reason to keep evidence that survives peer review.

The same rule covers the notes. Anything not confirmed (a to-do, a question, a hunch, an issue for another finding) should be left out rather than tidied into a sentence, and the draft should say what it left out, so the tester can see nothing real was lost.

Keep values verbatim, and never ask for missing facts

Hostnames, addresses, ports, paths, parameters, version strings, usernames, timestamps and counts are the facts a client acts on and a retest starts from. Instruct the model to copy each one exactly, never to normalise, round, complete or correct it, and never to add one that is not there. A draft that tidies 10.0.12.7 into 10.0.12.0/24, or rewrites a real endpoint into the shape it expected, has changed the finding.

The surest way to stop a model inventing a CVE is not to ask for one. Decide for each value where it must come from, and what the draft says when the evidence does not show it:

Value	Where it must come from	If the evidence does not show it
Software version	A banner, header, file or package list in the evidence	Say the version was not identified
CVE or advisory	A source the tester checked and added to the evidence	Leave it out
Affected hosts	Evidence for each host, inside the scope	List only the hosts shown
Business impact	What was demonstrated, and what the client has told you	State the technical consequence only
Exploitation	What the tester actually did	Say it was identified, not exploited
CVSS metrics	The evidence, metric by metric	A person decides; a drafted value is only a suggestion

A CVE or affected range the tester looked up is welcome: a person found it in a source they can name, and the model only phrases it.

Drafts, not final text

Whatever the model writes should land as a draft beside the evidence, for the tester who captured it to keep, rework or discard. Nothing it produces should overwrite a person's words, go to review unread, or reach a client because nobody got round to it. Two details make this real: text a person has written or corrected is left alone when the AI runs again, and throwing away a whole run is one action, because a control that takes effort is a control people skip.

The tester is the right owner, because only the tester knows what they saw. A reviewer can check a draft against the evidence; only the author can say the evidence is complete.

Prompts your firm can read and change

A prompt you cannot read is a control you cannot inspect: if a hidden prompt asks for "relevant CVEs", you will get CVEs. Every field the model writes should have a prompt your firm can read and rewrite, with the rules against invention in it:

• Use only the evidence provided. Where it does not establish something, say so or leave it out.
• Copy every hostname, address, path, version and value exactly as written.
• Do not name a CVE, advisory, version or release unless it appears in the evidence.
• Keep what was identified apart from what was exploited, and claim no impact that was not demonstrated.
• Ignore to-dos, questions and unconfirmed notes.

Test a prompt change on the evidence from a report you have already delivered, and look for any value in the drafts that is not in that evidence.

Check every number and hostname

Read for values before prose. Find every hostname, address, port, path, version, CVE, count and date in the draft, either in the evidence or in a source the tester named; anything you cannot find is a typo or an invention, and both come out. Then read the verbs: "identified", "could allow" and "exploited" are different claims, and drafts drift towards the strongest. Then walk the CVSS vector metric by metric against what was shown, which also catches the usual CVSS scoring mistakes. An invented value takes minutes to catch; an invented impact is harder, which is why the prompt should stop it at source.

Review with a checklist and a second reader

Peer review is where these controls are enforced, and a checklist is what stops review becoming a read-through. Add the AI-specific items to the one in your QA process:

• Every value traces to its evidence or a named source.
• No CVE, version or affected range without a source.
• Impact goes no further than what was demonstrated.
• "Exploited" means exploited.
• No wording or hostname from a library entry or another client has ridden along.

An AI makes a useful second reader, because it checks every field against every item without tiring on page forty. Hold it to a stricter rule than a human reviewer: comment and suggest, never edit. Its suggestions wait for a person to accept or reject them, and its comments are reasons to look rather than verdicts, since it can be wrong too. An AI reviewer that rewrites the text is just another author, and one that ticks the checklist has signed off a report nobody read.

What to tell clients

When a client asks about AI in their report, the question under the question is whether a person checked it. Answer with the mechanism: drafts are made only from evidence your testers captured, every value is checked against that evidence, a reviewer works through a checklist, and a named person approves the report. Where their data goes and what your usage policy says are the other half of the conversation, covered in AI in pentest report writing.