System Prompt Leakage
Users can extract the instructions given to the LLM, revealing internal rules, tool details and any sensitive data placed in them.
| Severity | Medium |
| CVSS 3.1 | 4.3 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N Open in the calculator |
| CVSS 4.0 | 5.3 CVSS:4.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:L/VI:N/VA:N/SC:N/SI:N/SA:N Open in the calculator |
| CWE | CWE-497: Exposure of Sensitive System Information to an Unauthorized Control Sphere |
| OWASP Top 10 | A01:2025 Broken Access Control |
| Category | Information Disclosure |
Description
The system prompt, also called the developer message or pre-prompt, holds the instructions that define how the application's large language model (LLM) behaves: its role, rules and restrictions, the tools it may call and sometimes reference data. Users can usually make the model reveal it, by asking it to repeat or translate the text above the conversation, to summarise its instructions, to output them in a code block or another encoding, or through role play. Instructions telling the model to keep its prompt secret are not a reliable control.
Disclosure matters because of what the prompt contains and how the application relies on it. Prompts found in testing often hold internal business rules, such as discount or approval limits, details of internal tools and APIs, names of staff or systems, and occasionally connection details or credentials that may give access to other services. An attacker who reads the prompt obtains any sensitive information it contains and learns how the assistant is configured, which makes prompt injection and other attacks easier to craft; where the prompt carries security rules, such as which users may see which data, it shows them exactly what to work around. Where the prompt contains only behavioural instructions, the direct impact is limited to assisting further attacks.
The weakness is confirmed by extracting the prompt through the conversation, often in several fragments, and checking that the result is consistent across attempts. A related and more serious variant is found by inspecting traffic: some applications build the system prompt in client-side code and send it with each request, where it can be read and also changed, so an attacker can replace it and remove the application's restrictions altogether.
Remediation
Assume that the system prompt will be read, and make sure that reading it gives an attacker nothing of value:
- Remove credentials, keys, connection strings, internal addresses and personal data from the prompt. Tools should authenticate in the back end with credentials the model never sees.
- Move security decisions out of the prompt: enforce authorisation, limits and business rules in application code and in the tools' back ends, so that knowing or bypassing an instruction changes nothing.
- Build the prompt on the server, and never accept it, or any part of it, from the client.
- Where the prompt has commercial value, accept that it cannot be fully protected. Output filters that detect prompt text, or a canary string embedded in the prompt, in responses can reduce casual extraction as defence in depth.
- Monitor conversations for extraction attempts.
Confirm the fix by repeating the extraction attempts, reviewing whatever prompt text can still be obtained for sensitive content, and checking client requests for prompt content.
References
Add This Finding to Your Library
In Pental this write-up is in Settings, Library. Add to my library copies it into your own, rated in your firm's CVSS version, ready to put in your own words.
Free trialYour trial starts when your portal goes live, not when you sign up, so the time you spend setting up does not count. · No card required
Only need the reporting? Our reporting tool is free and open source, with the same library.