Black Box, Grey Box or White Box: Which Test to Ask For

Black box, grey box and white box describe how much the tester is told before the test starts: nothing, accounts and documentation, or everything including source code. For most applications grey box finds the most per day, while black box, which buyers often ask for because it sounds more rigorous, is the most realistic simulation and the least efficient use of the days you are paying for.

The three, briefly

• Black box. The tester is given a name or an address and nothing else. No accounts, no documentation, no architecture.
• Grey box. Accounts for each role, and enough documentation to understand what the system is meant to do. No source code.
• White box. Everything, including source code and often a conversation with the developers.

Approach	Best at	Weakest at	Natural fit
Black box	Showing what an outsider with no head start can reach	Authorisation and business logic, which need the intended rules	Internet-facing perimeter
Grey box	Authorisation, logic and authenticated attack surface	Flaws only visible in code	Most web applications and APIs
White box	Subtle flaws in bespoke code, such as fail-open comparisons	Estates of products you did not write	High-consequence bespoke systems

What black box actually buys

Realism, and a genuine answer to one question: what can somebody with no starting advantage achieve? That is worth knowing, and for an internet-facing perimeter it is the natural framing.

What it costs is days. A meaningful part of a black box engagement goes on discovering what a grey box tester would have been handed on the first morning: which endpoints exist, what the roles are, how authentication works. On a five-day test that can be a day and a half of reconnaissance you already had the answers to.

Why grey box finds more

The classes of weakness that matter most in modern applications are about authorisation and logic, and both require knowing what should happen. A tester who does not know that a manager may approve refunds and a clerk may not cannot test that boundary, and no amount of probing reveals the intended rule.

Give a tester two accounts in each role and a paragraph explaining what each role is for, and the whole authorisation surface becomes testable, in the way set out in testing authorisation properly. That is the single biggest change you can make to what a test finds, and it costs you nothing but the accounts.

When white box is worth it

Source code access shifts the work again. It finds things that are nearly invisible from outside: a comparison that fails open on an unexpected input, a token generated from a predictable value, a permission check present on one route and absent on its sibling.

It is worth buying where the code is genuinely bespoke and where the consequences are high. It is less useful for an estate of standard products you did not write, and it needs a tester comfortable reading your stack, which is a narrower pool.

One caution: a white box test can drift into a code review, which is a different activity with a different rhythm. Agree which you are buying. A code review reads everything and reports what looks wrong; a test uses the code to aim, then proves the finding against a running system.

The words are not used consistently

There is no standards body policing these terms, and firms use them differently. Some call any authenticated test grey box; others reserve grey box for cases where documentation was supplied as well. A few use white box to mean full documentation without source code, which is what most people would call grey.

So do not buy the label. Write down what will actually be supplied: how many accounts in which roles, whether architecture documentation is provided, whether source code is available, and whether the tester can ask a developer questions during the engagement. Those four answers define the engagement completely and nobody can misread them.

What to hand over for a grey box test

A grey box test is only as good as what arrives on the first morning. In our experience the list below is what separates a test that starts at nine from one that starts on Wednesday, and preparing for a penetration test covers the rest of the groundwork.

• Two accounts per role, so one user's data can be requested as another's.
• A paragraph per role saying what it may and may not do, including the rules nobody wrote down.
• An API description if one exists, even an out-of-date one.
• Test data that looks like production in shape, with records owned by different users and tenants.
• A named developer who can answer a question within the hour.

The combination most people should buy

For an application: grey box, with two accounts per role, documentation, and an hour with somebody who can explain the intended rules. Add source access if the application is bespoke and the data is sensitive.

For a perimeter: black box is coherent, because the thing being tested is exactly what an outsider can see.

For an internal network: assumed breach, which is grey box by another name. Start the tester on a standard account and ask how far it goes.

What to write in the scope and the report

Whichever you pick, write down what was supplied and when, because it changes how the results should be read. Nothing found in a black box test of an application with an undocumented administrative interface means the interface was not discovered. The same sentence after a grey box test where that interface was named on day one means somebody looked at it. The scoping conversation is where this gets settled.

The report should also say what was not supplied. Ask for a coverage statement rather than only a findings list: which areas were exercised, which were not, and what stopped them. A reader a year later, deciding whether to trust the coverage, needs to know whether the tester had the map, and a firm that cannot produce that statement is reporting on what it happened to find.