When you open a conversation with an AI assistant, you think you're the first to speak. You're not. A text has already been sent to the model, invisible in the interface, telling it who it is, what it can do, what tone to adopt and what it must refuse. That's the system prompt, and it explains a good chunk of the behaviour that surprises you.
It's one of the least-known and most decisive layers of what you experience when using these tools. Let's break it down.
The principle
A raw language model doesn't know it's an assistant. It produces the most likely continuation of the text it receives, as we explained in our article on LLMs. Nothing in it says it should behave like a helpful interlocutor.
The system prompt fills that gap. It's a text placed at the top of the conversation, treated as a higher-level instruction, that sets the frame. It can be a few lines or several thousand words.
Its typical content covers four things: an identity and role, capabilities available such as web search or access to tools, behavioural guidelines on tone and format, and limits on what must be refused or handled with care.
Why it matters to you
Three common behaviours are explained by this layer.
Differences between interfaces. The same model accessed via a consumer app and via a developer API can behave differently. It's not the model that changes, it's the instructions framing it. A developer writes their own; an app user gets the provider's.
Refusals that seem arbitrary. When an AI declines a harmless request, it's often a broadly worded caution instruction triggering, rather than a decision by the model itself. This ties into what we described about guardrails: the dial is set somewhere, and it produces false positives.
Tone. Part of what we described as a model's personality comes from training, but another part comes from these instructions. It's the easiest layer to modify, and therefore the one providers tweak most often.
Should these instructions be made public? The debate is real. In favour: they determine what the tool agrees to do, and the user has a legitimate interest in knowing that. Against: publishing them makes them easier to bypass, since knowing the exact guidelines helps craft requests that slip past. Several players have chosen to publish all or part of theirs, which aligns with the transparency we called for regarding impossible neutrality. Others haven't.
What it changes if you're building
For anyone building a product on an API, the system prompt is the main lever for customisation, ahead of even the model choice.
Three practical tips. Be precise about the role and scope: a vague frame produces off-topic responses. Place these instructions at the top and keep them stable, which lets you benefit from caching and cut costs significantly. And above all, never treat these instructions as a security barrier: they steer behaviour, they don't guarantee it. Malicious content in the data your system processes can contradict them, as shown by the agentjacking mechanism.
What to remember
There's always someone who wrote something before you. That's perhaps the best way to sum up this layer.
That doesn't mean you're being manipulated: these instructions most often serve to make the tool useful and safe. But they constitute an invisible editorial choice, deciding what the machine will agree to do for you. Knowing they exist changes how you interpret a refusal, a difference in tone, or unexpected behaviour. It wasn't the machine that decided: it was someone, somewhere, who wrote a line.