Configuration Review or Penetration Test: Which to Choose

A configuration review checks whether an environment is set up the way good practice and the client's own policy say it should be; a penetration test shows what an attacker in a stated position can actually achieve. A client asking for a penetration test of their cloud environment often needs the first, sometimes needs both, and will compare your quote against whichever one the other firm decided to offer.

The distinction is not pedantry. The two produce different findings, take different time, and answer questions that only overlap in the middle.

Two different questions

A configuration review asks whether the environment is set up the way good practice and the client's own policy say it should be. It is largely an authenticated, read-oriented exercise against the control plane, and it is thorough by nature: you can enumerate every bucket, every role, every rule.

A penetration test asks what an attacker in a stated position can actually achieve. It is goal-oriented rather than exhaustive, it chains weaknesses that are individually unremarkable, and it stops being useful the moment it turns into a checklist.

The reason both exist is that neither subsumes the other. A perfectly configured environment can still be compromised through the application running in it. A badly configured one can survive a time-boxed test simply because the tester spent their days elsewhere.

What a configuration review answers well

• Coverage. Every resource of a given type, not the ones that happened to be reachable.
• Drift. What has changed against a baseline, which is a question a test cannot answer at all.
• Policy alignment. Whether the environment matches what the organisation says it requires.
• The unglamorous majority: logging that is off, retention that is not set, a role with a wildcard that nobody has looked at since it was created.

What only a test answers

• Whether the misconfiguration is reachable, and by whom.
• What chains. Two acceptable settings and one weak application often make a path that neither review would flag.
• Whether the detection the client believes in produces an alert that reaches a human.
• What an attacker gets, expressed as consequences rather than as control names.

That last one is why buyers ask for a test when they mean a review. A review produces a list of settings; a test produces a sentence about customer data. Executives fund the sentence.

Where the wrong one gets sold

Two failure shapes, and both are avoidable in the scoping call.

The first is a "cloud penetration test" that is a configuration review with an aggressive title, delivered as a list of policy deviations with no demonstrated path. The client believes they have been attacked and survived. They have not been attacked at all.

The second is a genuine test, correctly scoped and time-boxed, that the client reads as coverage. Three days against a large estate is a sample, and a clean report means the tester did not find a path in three days. Saying that plainly in the report is the difference between an honest deliverable and one that will be misquoted in a board pack.

Scoping a hybrid honestly

Most clients are best served by both, and there is nothing wrong with selling both provided the boundary is stated. What matters is that the days are separated and the report keeps them separate. Check the provider's own testing rules before either starts; cloud penetration testing rules sets out what the major providers permit.

Run the review first where you can. It surfaces the reachable weaknesses that make the test days productive, and it stops the test being spent rediscovering an open storage bucket that an enumeration would have handed you on the first morning.

Where the budget only stretches to one, the question that decides it is what the client will do with the answer. An organisation that has never looked at its configuration should look at its configuration. An organisation with a mature baseline and a specific fear about an application should have the application tested.

Make the difference legible in the report

A year later, somebody who was not in the scoping call will read this document and draw a conclusion from it. Write for that reader, and be ready for the question covered in proving what you tested when a client asks.

• State which activity produced each finding, especially where the engagement was both.
• State the position the tester started from: unauthenticated from the internet, an assumed-breach foothold, a read-only role on the control plane.
• State what was not covered, and whether that was scope or time.
• Where the finding is a deviation from policy rather than a demonstrated path, say so. Both are worth reporting; conflating them is what makes a report unusable as evidence.

The third thing clients often mean

Sometimes the request is for neither. A client asking for a "penetration test" to satisfy a customer questionnaire or an insurer frequently needs a vulnerability scan with a human interpreting the output, and there is nothing dishonest about selling exactly that, provided it is what the document says.

The dishonesty is in the naming. An authenticated vulnerability scan with the false positives removed is a useful, affordable, defensible deliverable. Called a penetration test, it becomes the thing a client waves at their largest customer, and it will not survive the first competent question about methodology.

Ask what the report has to satisfy. If the answer is a specific clause in a contract or a framework, read the clause. It frequently asks for less than the client assumes, as penetration testing for ISO 27001 and SOC 2 shows, and occasionally for something quite specific that a generic engagement will not provide.

What a clean report actually means

Every time-boxed engagement can end with nothing significant found, and how you write that determines whether the client draws a true conclusion or a comforting one.

A clean report means the tester did not find a path in the days available, from the position agreed, within the scope defined. It does not mean the estate is secure, and a report that does not say so will be read as if it does. Put the sentence in, in the summary, in plain words: what was covered, from where, for how long, and what that permits the reader to conclude.

The same sentence protects you. When something happens later in an area nobody was asked to look at, the difference between a report that stated its limits and one that did not is the difference between a difficult conversation and a serious one.