System prompt vs user prompt (ChatGPT & production templates)

A system prompt (durable instructions) sets role, rules, and output contracts for the model; a user prompt carries the per-request task, data, and questions. Field names differ by vendor — OpenAI Responses instructions / developer, Chat Completions system/developer, Claude system, Gemini systemInstruction — but the split of stable contract vs volatile task does not.

Maker ship path: tighten the draft in the prompt formatter / cleaner → measure prefix cost with the token counter & cost calculator → score which vendor/seat you will pin for launch week in the AI tool comparison checklist → version the prompt in git before you ship.

This guide covers the difference, generic system prompt examples you can adapt, vendor channel facts re-verified 2026-09-22, and how to keep production prefixes short, versioned, and eval’d. No proprietary leaks.

System prompt vs user prompt

Rule of thumb: if changing it requires a PR and an eval, it belongs in the durable channel (system / developer / instructions / systemInstruction). If it changes every request, it belongs in the user message.

Vendor channels (verified 2026-09-22)

Re-check primary docs before you freeze an integration — APIs move.

OpenAI prompt objects: reusable API prompt objects (v1/prompts) are being de-emphasized; creation de-emphasized from 2026-06-03, shutdown scheduled 2026-11-30. Store production prompts in code + git, not only a vendor dashboard. See OpenAI’s text-generation “Version prompts in code” guidance.

Practical system prompt examples (generic)

These are illustrative templates, not leaked vendor or product prompts. Adapt names, schemas, and refusals to your domain. Paste into instructions / system / systemInstruction — not into the user turn.

1) Support assistant (JSON ticket triage)

# Role
You are a support triage assistant for Acme App. You classify and draft replies; you do not invent account balances or invoice IDs.

# Scope & refusals
- Only discuss Acme App product and billing topics present in CONTEXT.
- If asked for legal, medical, or tax advice: refuse briefly and suggest a qualified professional.
- If CONTEXT is missing required fields: ask at most 2 clarifying questions OR return nulls per schema.

# Output contract
Respond with ONLY valid JSON:
{
  "intent": "billing|bug|how_to|other",
  "priority": "low|medium|high",
  "reply_draft": "string",
  "needs_human": true|false
}

# Style
Clear, concise, no emojis unless the user used them first.

2) ChatGPT-style coding helper (repo-scoped)

# Role
You are a coding assistant for the user's current repository. Prefer small, reviewable diffs.

# Scope
- Follow existing project conventions when shown in CONTEXT.
- Do not invent APIs, env vars, or files that are not in CONTEXT.
- If a change is risky (migrations, auth, payments): list risks before code.

# Output
1. Short plan (≤5 bullets)
2. Code blocks with paths
3. Test / verify steps the user can run

# Refusals
Refuse requests to bypass auth, exfiltrate secrets, or generate malware.

3) Structured extraction (documents → schema)

# Role
Extract fields from the provided document text. Do not invent values.

# Output contract
Return ONLY JSON matching SCHEMA. Use null for unknown fields. Never guess IDs or dates.

# Rules
- Copy strings verbatim when possible.
- Normalize dates to ISO-8601 only when the source is unambiguous; otherwise null.
- If the document is not relevant to SCHEMA, return {"error":"unrelated_document"}.

4) Minimal “best system prompt” skeleton

# Role
…

# Scope & refusals
…

# Output contract
…

# Tools (if any)
Allowed: … | Do not call when … | On tool error: …

# Style
… (short)

# Missing information
Ask at most N clarifying questions OR fill with nulls per schema.

Ship-week loop: draft → prompt cleaner → token estimator (prefix × expected daily calls) → pin model/seat in comparison checklist → PR the prompt file.

What belongs in the system prompt

Put durable rules here:

• Role and scope (“you are a support assistant for Product X; you do not invent pricing”)
• Output contracts (JSON schema, markdown sections)
• Safety and refusal policy specific to your domain
• Tool-use rules (when to call, when to ask the user)
• Tone constraints that rarely change

Keep volatile task details in the user message or a retrieved brief. Bloated system prompts raise cost on every call and make diffs scary.

Write it as a contract

Prefer normative language:

You MUST respond with valid JSON matching SCHEMA.
You MUST NOT invent invoice IDs.
If the user asks for legal advice, refuse and suggest contacting counsel.

Avoid soft mush: “Try to be helpful and mostly stick to JSON when possible.”

Tool-use section

If the model can call tools, spell out:

1. Allowed tools and one-line purpose each
2. When not to call (answer from provided context first)
3. How to handle tool errors (retry once vs surface error)
4. Parallel vs sequential calling policy

Ambiguous tool policy → wasted calls → latency and cost. For tool-call token overhead, see AI agent tool-calling token overhead.

Versioning and rollout

1. Store prompts in git (prompts/support-v4.txt or a typed config module) — not only a vendor UI
2. Change with a PR description: behavior delta + eval notes
3. Shadow-test on logged traces before 100% traffic
4. Keep a kill switch to pin the previous version
5. On OpenAI Responses: resend instructions every turn if you rely on previous_response_id (they do not auto-carry)

Never edit production prompts only in a vendor dashboard with no history.

Eval the prompt, not your gut

Maintain a small golden set (20–50 cases):

• Happy paths
• Missing fields
• Jailbreak-ish asks relevant to your product
• Multilingual if you support it
• Long context / noisy retrieval

Score with assertions (schema valid, required keys present, banned phrases absent). LLM-as-judge can supplement; it should not be the only gate for safety-critical behavior.

Cost and latency notes

Every token in the durable prefix is paid on every request. Periodically:

• Delete dead rules
• Move rarely needed policy into conditional prefixes
• Prefer references (“follow POLICY.md section 2”) only if that text is actually in context — dangling references confuse models

Budget the prefix before launch: paste the system text into the token estimator with editable $/1M rates, then multiply by expected daily calls.

FAQ — system prompts (2026-09-22)

What is the difference between a system prompt and a user prompt?

The system (or developer / instructions / systemInstruction) channel holds durable product rules; the user channel holds the per-request task and data. If a change needs a PR and an eval, it is system-side.

Where do I put system prompts in OpenAI, Claude, and Gemini today?

OpenAI: Responses instructions or developer role (preferred for new apps); Chat Completions still accepts system/developer. Claude: Messages API system. Gemini: systemInstruction / system_instruction (re-send on Interactions when chaining). Verify vendor docs the day you ship.

How do makers keep system prompts from blowing up token cost?

Keep the prefix short, version it in git, clean drafts in prompt-cleaner, estimate burn in token-estimator, and freeze one model/seat via comparison-checklist before launch week.

Related

• Prompt template library for content ops
• Document prompt changes like config
• JSON schema output prompts for production
• Prompt injection defenses for customer bots
• Prompt formatter & cleaner — iterate system prompts without signup
• Token counter & cost calculator — budget the always-on prefix
• AI tool comparison checklist — pin the vendor before you ship