Local models for finding write-ups: what they are good at

Writing up a finding is unavoidable, repetitive, and the same twelve vulnerabilities every quarter in slightly different words. It is an obvious candidate for a model. It is also an obvious place to be careful, because the output goes out under your name with your methodology attached.

The argument for local that is not about privacy

Local inference usually gets framed as a compliance compromise: slower and worse, but the data stays in. For this workload that framing is wrong, and the reason has nothing to do with privacy.

Hosted providers give you rate limits. Rate limits push a product towards fewer, larger, faster calls, which is exactly the wrong shape for careful drafting. On hardware you own there is no per-minute ceiling, so the work can afford several slower passes: draft, check it against the evidence, revise. When a human is going to review the output either way, slower and better beats faster and thinner.

The privacy argument is real as well, and it is simpler than it is usually made. A finding is a working description of how to get into a named client. Sending it to a third party is a decision, and it is one your client is entitled to be told about.

What the hardware needs to be

The binding constraint is memory, not compute. A model that fits entirely in VRAM is usable. One that spills into system memory is roughly an order of magnitude slower, and a feature that slow gets switched off.

• 8 GB: a small model. Fine for individual finding fields, weak on long summaries.
• 16 GB: the practical sweet spot for report drafting.
• 24 GB and up: longer context, whole-assessment summaries, comfortable headroom.
• Apple silicon: unified memory works well, the total is what matters.
• CPU only: technically works, too slow to use in anger.

Context window is the lever people get wrong. Setting it far beyond what the hardware holds does not fail loudly, it silently spills and everything becomes slow. Set it to what the machine can actually hold, measure, then raise it.

Good at, not good at

Good at: turning notes and evidence into a first draft of description, impact and remediation in a consistent house voice. Rewriting a familiar class of finding for a new client without dragging the previous client’s specifics across. Summarising what changed between a test and its retest. Writing an executive summary from the findings actually present rather than a generic preamble about the threat landscape.

Not good at: severity. A model will produce a confident CVSS vector and it will be wrong in ways that are hard to catch, because the shape of the answer is right. Severity is a judgement about a specific environment and it belongs to the tester. Same for exploitability claims and anything that reads as an assurance.

The rule that survives review: a model may draft anything a client will read, and decide nothing a client will rely on.

Keeping it defensible

• Generated text goes through the same QA gate as hand-written text. If the gate is skippable for AI drafts, you do not have a gate.
• Prompts live in your configuration, versioned and editable, not hidden inside a product. When a client asks how a paragraph was produced you want to be able to show them.
• Feed it the evidence, not the title. A summary written from findings is checkable; one written from a heading is fiction with good grammar.
• Write down what the model is never allowed to author. Severity, scope statements and attestation wording is the usual list.

An afternoon that settles it

Run a model on one workstation and draft the finding fields for an assessment you have already delivered. Compare against what you actually shipped. That comparison is the only benchmark that matters, it takes an afternoon, and it tells you more than any evaluation table.

If the drafts are close, the saving is real and recurring. If they are not, you have learned something specific about where the gap is, which beats an opinion about AI in general.