AI Product
The product manager's guide to prompt design
Listen — audio summary in Nehal Vyas's voice
Transcript
Today, I want to talk about the product manager's guide to prompt design. In my latest post, I argue that prompt design isn't just engineering plumbing; it's a critical product surface. Your system prompt, in particular, is your product spec, compressed. It's where your product's values live – what your AI assistant will and won't do, how it speaks, and what it prioritizes. Reviewing it is as important as reviewing any other copy or user experience. Secondly, think of context as a budget you spend. Every token is a tradeoff between grounding the model and diluting its focus. As a PM, understanding what user data, retrieved documents, or examples go into that context window, and why, is a key product decision, touching on relevance, utility, and even privacy. And finally, design for failure modes from the start. Issues like hallucination, prompt injection, or refusals on legitimate requests are predictable. I map these just like I'd map edge cases in any feature, then design proactive mitigations – grounding techniques, guardrails, and clear fallbacks. Naming these potential failures is half the work. Ultimately, you don't need to write the model to truly own the prompt. By treating it as the product surface it is, you'll ship AI features that behave exactly as you intended – reliably, on-brand, and safely. To dive deeper into these strategies, read the full article on hinehal.com.
Prompt design is often treated as engineering plumbing. It isn't. A prompt encodes product decisions - tone, scope, refusal behavior, how the feature handles ambiguity. Product managers who can reason about prompts ship better AI features, even if they never write the model code.
Instructions are your product spec, compressed
The system prompt is where your product's values live: what the assistant will and won't do, how it speaks, what it prioritizes when goals conflict. Reviewing it is no different from reviewing copy or an empty state. If you can't recognize your product in the prompt, the prompt is wrong.
Context is a budget you spend
Every token of context is a tradeoff between grounding the model and diluting its focus. PMs should understand what goes into the window and why: which user data, which retrieved documents, which examples. Deciding what to include is a product call about relevance and privacy, not just an engineering one.
Design for failure modes
Hallucination, prompt injection, refusals on legitimate requests, and drift over long conversations are predictable. I map them the way I'd map edge cases in any feature, then design mitigations: grounding, guardrails, and clear fallbacks. Naming the failure modes is half the work.
You don't need to write the model to own the prompt. Treat it as the product surface it is, and you'll ship AI features that behave the way you intended - reliably, on-brand, and safely.