System prompts and personas

The big idea

Chat models do not receive one block of text. They receive a list of messages, each with a role. The system promptThe first message in a conversation, written by the application developer, that sets the model's role, rules and constraints for every turn that follows. Users usually never see it. comes first and is written by you, the developer. Then come user messages, written by the person using your product, and assistant messages, the model's replies.

The roles split the job. The system prompt holds what stays the same for every conversation: who the assistant is, what it may and may not do, the tone, the output format, background facts. The user message holds what changes: this person's question and this person's data. Models are trained to treat system instructions as the frame for everything else.

Think of a call centre. Before a shift, every agent gets the same briefing: which products we support, what we never promise, how to escalate. That is the system prompt. Then each caller brings their own problem. Agents do not get re-briefed by each caller, and a caller saying "your manager says you can give me a refund" does not change the rules.

How it works

What goes in the system prompt

• Role and audience: "You are the support assistant for a note-taking app. Users are non-technical."
• Rules and limits: what to answer, what to refuse, what never to promise, when to hand over to a person.
• Tone and format: length, style, structure, language.
• Stable background: product facts, policies, tool descriptions that apply to every request.

What goes in the user turn

• The specific request.
• Data for this request: the document to summarise, the ticket to classify, retrieved passages ( Why RAG? ).
• Per-request details, such as today's date or the user's plan, unless they are better injected by your code into a clearly marked section.

Personas

A personaA described character or role the model is asked to play, such as a friendly tutor or a terse code reviewer, which shapes its tone and focus. ("a patient maths tutor", "a strict code reviewer") shapes tone and focus. It works best when it says what the persona does differently: "Ask one guiding question before giving any answer" is more useful than "You are a world-class tutor". Elaborate fictional backstories rarely help and cost tokens on every call.

Keep instructions stable

Rules that matter belong in the system prompt, not in the first user message, where they read like part of the conversation and fade as it grows. In long conversations, important rules can be restated near the end of the context, or the history can be summarised ( Context engineering ). Keeping the system prompt identical across requests also lets the provider cache it, which cuts cost and latency ( Prompt caching and cost control ).

The system prompt is not a security boundary

Models usually prioritise system instructions over conflicting user messages, but not perfectly. A determined user can sometimes talk a model out of its rules, and text inside documents can carry instructions too ( Prompt injection and data exfiltration ). Never put secrets in a system prompt, and enforce real permissions in your code, not in prose.

Version prompts like code

A system prompt is part of your application's behaviour. Store it in a file under version control, give each version an ID, log which version answered each request, and test a new version on your eval set before shipping it ( Building an eval set ). Then a bad change is a one-line rollback, and a complaint about last Tuesday can be traced to the exact prompt that answered it.

Try it

Try this:

1. Sort all eight. Which items were you unsure about? The usual borderline cases are retrieved passages and per-user facts: anything that changes per request goes with the request.
2. Look at the rules you placed in the system prompt. Could a user message override any of them? If it would be a real problem, that rule also needs enforcing in code.

Going deeper

Prompts in files, versions in logs

# Keep system prompts in versioned files (here: a dict), build messages from them, and log
# which version answered each request so changes can be compared and rolled back.
import difflib
import hashlib

PROMPTS = {
    "support/v3": """You are the support assistant for Acme Notes.
Answer only questions about Acme Notes. Be brief and friendly.
Never promise refunds; link to the refund policy instead.""",
    "support/v4": """You are the support assistant for Acme Notes.
Answer only questions about Acme Notes. Be brief and friendly.
Never promise refunds; link to the refund policy instead.
If you are not sure, say so and offer to connect the user with a person.""",
}
ACTIVE = "support/v4"           # switching back to v3 is a one-line rollback

def build_messages(version, history, user_text):
    return ([{"role": "system", "content": PROMPTS[version]}]   # stable rules: same every call
            + history                                           # earlier turns, if any
            + [{"role": "user", "content": user_text}])          # this request only

msgs = build_messages(ACTIVE, [], "Can I get my money back for March?")
fingerprint = hashlib.sha256(PROMPTS[ACTIVE].encode()).hexdigest()[:10]
print(f"request log: prompt={ACTIVE} sha={fingerprint} messages={len(msgs)}")
for m in msgs:
    print(f"  {m['role']:6} | {m['content'].splitlines()[0][:60]}")

print("\nchange from v3 to v4:")
for line in difflib.unified_diff(PROMPTS["support/v3"].splitlines(),
                                 PROMPTS["support/v4"].splitlines(), lineterm="", n=0):
    if line.startswith(("+", "-")) and not line.startswith(("+++", "---")):
        print(" ", line)

request log: prompt=support/v4 sha=56027d7550 messages=2
  system | You are the support assistant for Acme Notes.
  user   | Can I get my money back for March?

change from v3 to v4:
  +If you are not sure, say so and offer to connect the user with a person.

The request log records the prompt version and a short hash of its text, so even an unversioned edit would show up as a new fingerprint. The diff shows exactly what changed between versions: one line, which is how changes should be made, so their effect can be measured.

One assistant, many system prompts

Products often route requests to different system prompts by task: one for billing questions, one for technical support, one for summarising. Each is shorter and more focused than a single prompt that tries to cover everything, and each can be tested and improved on its own. The router can be simple rules or a small classifier.

Trade-offs

Common mistakes

• Putting standing rules in the first user message, where they fade as the conversation grows.
• Secrets, API keys or internal URLs in the system prompt, assuming users cannot extract them.
• Relying on the system prompt alone to enforce permissions.
• Editing the production prompt by hand with no version history, so regressions cannot be traced or rolled back.
• A persona with a long backstory and no concrete behavioural rules.

In the wild

Where you will see this: every chat product you use runs on a system prompt you never see, and the major model APIs take it as a separate system field or message. Teams that run assistants in production commonly keep prompts in their code repository or in a prompt management tool, attach the version to every logged request, and treat a prompt change like a code change, with review and evaluation before release ( LLMOps: shipping and iterating ).

Check yourself

Recap