Black Box, Grey Box or White Box: Which Test to Ask For

Buyers often ask for black box because it sounds more rigorous. It is the most realistic simulation and, for most purposes, the least efficient use of the days you are paying for.

The three, briefly

• Black box. The tester is given a name or an address and nothing else. No accounts, no documentation, no architecture.
• Grey box. Accounts for each role, and enough documentation to understand what the system is meant to do. No source code.
• White box. Everything, including source code and often a conversation with the developers.

What black box actually buys

Realism, and a genuine answer to one question: what can somebody with no starting advantage achieve? That is worth knowing, and for an internet-facing perimeter it is the natural framing.

What it costs is days. A meaningful part of a black box engagement goes on discovering what a grey box tester would have been handed on the first morning: which endpoints exist, what the roles are, how authentication works. On a five-day test that can be a day and a half of reconnaissance you already had the answers to.

Why grey box finds more

The classes of weakness that matter most in modern applications are about authorisation and logic, and both require knowing what SHOULD happen. A tester who does not know that a manager may approve invoices and a clerk may not cannot test that boundary, and no amount of probing reveals the intended rule.

Give a tester two accounts in each role and a paragraph explaining what each role is for, and the whole authorisation surface becomes testable. That is the single biggest change you can make to what a test finds, and it costs you nothing but the accounts.

When white box is worth it

Source code access shifts the work again. It finds things that are nearly invisible from outside: a comparison that fails open on an unexpected input, a token generated from a predictable value, a permission check present on one route and absent on its sibling.

It is worth buying where the code is genuinely bespoke and where the consequences are high. It is less useful for an estate of standard products you did not write, and it needs a tester comfortable reading your stack, which is a narrower pool.

One caution: a white box test can drift into a code review, which is a different activity with a different rhythm. Agree which you are buying. A code review reads everything and reports what looks wrong; a test uses the code to aim, then proves the finding against a running system.

The words are not used consistently

There is no standards body policing these terms, and firms use them differently. Some call any authenticated test grey box; others reserve grey box for cases where documentation was supplied as well. A few use white box to mean full documentation without source code, which is what most people would call grey.

So do not buy the label. Write down what will actually be supplied: how many accounts in which roles, whether architecture documentation is provided, whether source code is available, and whether the tester can ask a developer questions during the engagement. Those four answers define the engagement completely and nobody can misread them.

What each one implies for the report

The framing changes what an absence of findings means, and the report should say so. Nothing found in a black box test of an application with an undocumented administrative interface means the interface was not discovered. The same sentence after a grey box test where it was named on day one means somebody looked at it.

Ask for a coverage statement rather than only a findings list: which areas were exercised, which were not, and what stopped them. A firm that can produce that has thought about the difference; one that cannot is reporting on what it happened to find.

The combination most people should buy

For an application: grey box, with two accounts per role, documentation, and an hour with somebody who can explain the intended rules. Add source access if the application is bespoke and the data is sensitive.

For a perimeter: black box is coherent, because the thing being tested is exactly what an outsider can see.

For an internal network: assumed breach, which is grey box by another name. Start the tester on a standard account and ask how far it goes.

What to write in the scope

Whichever you pick, write down what was supplied and when, because it changes how the results should be read. A report saying no critical findings after a black box test on a system with an undocumented administrative interface is a different statement from the same sentence after a grey box test where that interface was named on day one.

The report should also say what was NOT supplied. A reader a year later, deciding whether to trust the coverage, needs to know whether the tester had the map.