Prompt engineer interview questions

As asked

A support assistant needs conversation history, policy docs, customer metadata, and tool results, but the context window is regularly exceeded. How do you redesign the prompt flow?

Sample answer outline

The answer should treat the context window as a scarce product resource. Keep durable instructions short, summarise long conversation history with provenance, retrieve only task-relevant policy chunks, and pass customer metadata as a compact structured block rather than prose. Tool results should be schema-shaped and pruned to fields the next step needs. Add tests for truncation boundaries because prompt changes can silently push out important context. Candidates often rely on larger context windows instead of deciding what information is actually needed for the task.

Expect these follow-ups

• How do you know a summary has not lost a critical fact?
• What context should never be summarised?
• How would you debug a failure caused by truncation?

As asked

A product team wants a generative assistant to answer support questions in the company's tone using private documentation. When would you fine-tune, use RAG, or rely on prompting?

Sample answer outline

Use RAG when the main requirement is fresh private knowledge and citations, because fine-tuning is a poor way to memorise changing facts. Use fine-tuning when you need a consistent behaviour, format, style, or task policy that prompting cannot reliably maintain at scale. Prompting alone is suitable when the task is simple, low-risk, and changes frequently. Strong answers mention evaluation before and after each approach, plus the operational costs of data curation, retraining, retrieval quality, and latency. The common error is treating fine-tuning as a magic upgrade rather than a targeted behavioural intervention.

Expect these follow-ups

• What failure tells you the retriever is the problem?
• What data would you need for a useful fine-tune?
• How would you measure whether tone improved without making factuality worse?

As asked

When you're designing a new prompt, how do you decide what goes in the system prompt versus the user turn? Walk me through your reasoning with a concrete example.

Sample answer outline

Strong answers distinguish persistent persona/context/constraints (system) from dynamic per-request content (user turn). They note that some models treat system content as higher-trust, that leaking instructions into the user turn can be overridden or confused with user content, and that token economy matters when the system prompt is cached. A concrete example such as a customer-support bot shows the reasoning clearly.

Expect these follow-ups

• How does your answer change if the model doesn't support a distinct system role?

As asked

You need to add few-shot examples to a classification prompt. What criteria do you use to select which examples to include, how many to use, and in what order?

Sample answer outline

A strong answer covers: diversity across classes and edge cases, choosing examples representative of hard boundaries rather than easy cases, ordering effects (recency bias means the last example often has the most influence), keeping example count proportional to context budget, and validating example selection empirically on a held-out eval set rather than guessing.

Expect these follow-ups

• If you had 500 candidate examples and needed to pick 8, what process would you use?

As asked

Chain-of-thought prompting often improves accuracy, but it isn't always the right choice. When would you deliberately leave it out, and what are the practical downsides of always enabling it?

Sample answer outline

Good answers mention: latency and token cost on high-QPS endpoints, leaking internal reasoning to users (CoT is visible in the response), cases where the answer is retrievable from memory without decomposition, risk that incorrect intermediate steps propagate (so CoT can hurt on factual recall tasks), and the option to use scratchpad/thinking tokens that are not returned to the user.

Expect these follow-ups

• How do you evaluate whether CoT is actually helping accuracy on your task?

As asked

You need an LLM to return valid JSON with a specific schema every time. What techniques do you combine to achieve near-100% schema compliance, and what do you do when the model still fails?

Sample answer outline

Strong candidates mention: constrained decoding or grammar-based sampling (JSON mode, tool-calling APIs), clear schema description in the prompt with an example, asking the model to emit the key names in a fixed order, retry logic with error feedback, post-processing validation with a schema library, and fallback parsing that extracts partial JSON. They also note that asking the model to check its own output before returning can catch many errors.

Expect these follow-ups

• How does your approach change if the API has no native JSON mode?