Running AI report writing on your own hardware

Drafting a finding is the part of the job that is both unavoidable and least interesting. It is also the part where the same twelve vulnerabilities get described from scratch every quarter in slightly different words. That makes it an obvious candidate for a model, and an equally obvious place to be careful, because the output goes out under your name with your methodology attached to it.

Why local is a real option here

Local inference is often dismissed as a compliance compromise. For this particular workload it is frequently the better engineering choice, for a reason that has nothing to do with privacy.

A hosted provider gives you rate limits. Rate limits push a product towards fewer, larger, faster calls, which is exactly the wrong shape for careful drafting. On your own hardware there are no per-minute ceilings, so the work can take several slower passes: draft, check against the evidence, revise. Slower and better beats faster and thinner when a human is going to review it either way.

The privacy argument is real too. A finding is a working exploit description against a named client. Sending it to a third party is a decision, and it is one your client is entitled to be told about. Keeping it inside your network means the answer to that question in your next supplier assessment is one sentence long.

What the hardware actually needs to be

The binding constraint is memory bandwidth and VRAM, not raw compute. A model that fits entirely in VRAM is usable; one that spills into system memory is roughly an order of magnitude slower and will make the feature feel broken.

Context window is the other lever and the one people get wrong. Setting it far larger than the hardware supports does not fail loudly; it quietly spills and everything becomes slow. Set it to what the machine can actually hold and raise it once you have measured.

What it is good at, and what it is not

Good: turning notes and evidence into a first draft of a description, an impact statement and remediation guidance in a consistent house voice. Rewriting the same class of finding for a different client without copying the previous client’s specifics across. Summarising what changed between an initial test and a retest. Producing an executive summary that reflects the findings actually present rather than a generic preamble about the threat landscape.

Not good: deciding severity. A model will produce a confident CVSS vector, and it will be wrong in ways that are hard to spot because the shape of the answer is right. Severity is a judgement about a specific environment and it belongs to the tester. The same goes for exploitability claims and anything that reads as an assurance.

The rule that holds up in review: a model may draft anything a client will read, and decide nothing a client will rely on.

Keeping the output defensible

• Every generated field goes through the same human review gate as a hand-written one. If your QA step is skippable for AI-drafted text, you do not have a QA step.
• Prompts belong in your configuration, versioned and editable, not hidden inside a vendor’s product. When a client asks how a paragraph was produced, you want to be able to show them.
• Feed it your evidence, not your assumptions. A summary written from the findings actually in the assessment is checkable; one written from a title is fiction with good grammar.
• Keep a house style and enforce it in the prompt, so the drafts arrive already sounding like your firm rather than needing to be rewritten into it.
• Decide, in writing, what the model is never allowed to author. Severity, scope statements and attestation wording are the usual list.

Getting started without committing

Run a local model on one workstation and point the portal at it. Draft the finding fields for one completed assessment you have already delivered, then compare against what you actually shipped. That comparison is the only benchmark that matters, it takes an afternoon, and it will tell you more than any evaluation table.

If the drafts are close, the time saved is real and recurring. If they are not, you have spent an afternoon and learned something specific about where the gap is, which is a better position than an opinion about AI in general.