Prompt engineer technical fundamentals interview questions

As asked

A stakeholder wants to improve the model's performance on a specialized task. How do you decide between prompt engineering with few-shot examples versus fine-tuning the model?

Sample answer outline

Key factors: prompt engineering is faster to iterate, has no training cost, and requires no labeled data, but is limited by context length and is less sample-efficient at capturing format or style. Fine-tuning using parameter-efficient methods (LoRA, QLoRA, prefix tuning) can work with as few as a few dozen to a few hundred examples, far fewer than classic full fine-tuning required. Full fine-tuning may need thousands. Fine-tuning wins when: consistent output format is critical, the task behavior is hard to express in words but easy to show in examples, or you need to reduce inference latency by offloading instructions to weights. Prompt engineering wins for rapid iteration, low-resource tasks, and when requirements change often. The answer should also mention they are not mutually exclusive: you can fine-tune a model and still use prompt engineering on top of it.

Expect these follow-ups

• At what data scale does fine-tuning typically start to outperform few-shot prompting?

As asked

Lay out the conditions under which zero-shot prompting outperforms few-shot, and the conditions under which adding chain-of-thought to few-shot further improves things. What determines which regime you're in?

Sample answer outline

Zero-shot wins when: the task is well within the model's training distribution, examples would just add noise or bias the output format, and latency or token cost makes examples prohibitive. Few-shot wins when: the output format is non-standard, the task has nuance that is hard to describe but easy to demonstrate, or the model needs calibration on label definitions. CoT on top of few-shot wins on multi-step reasoning tasks where the intermediate steps are meaningful. The determining factor is how much of the task can the model infer from its weights alone versus what it needs shown.

Expect these follow-ups

• Does few-shot help more on smaller or larger models, and why?

As asked

At a conceptual level, what does self-attention compute, and why does it matter for prompting? What aspect of attention explains why position in the context window affects how well information is recalled?

Sample answer outline

Self-attention computes weighted sums of value vectors where weights are similarity between query and key vectors. This means the model can attend to any position in the context, but attention patterns learned during training may concentrate on certain positions (beginning and end) more than the middle for very long sequences. Good answers connect theory to prompt design implications without needing to derive the math.

Expect these follow-ups

• Does a longer context window always mean better performance? Why or why not?

As asked

How does reinforcement learning from human feedback change how a base model responds to instructions, and what side effects does it introduce that a prompt engineer needs to account for?

Sample answer outline

RLHF (and RLAIF variants) train the model to produce outputs that human raters prefer, which instills instruction following, helpfulness, and safety. Side effects: sycophancy (model agrees with the user's framing even when wrong), verbosity bias (longer rater-preferred responses), over-caution (erring toward refusal), and occasional reward hacking. Prompt engineers should account for sycophancy by not leading questions with an opinion and testing robustness to user disagreement.

Expect these follow-ups

• How does sycophancy affect your eval design?

As asked

How does BPE tokenization work at a high level, and what practical effects does tokenization have on prompt design, especially for numeric data or unusual words?

Sample answer outline

BPE merges frequent character pairs into subword units. Practical effects: numbers are often split into individual digit tokens (causing arithmetic errors), rare or domain-specific words are fragmented (prompting the model to spell them out helps), whitespace and punctuation affect token boundaries (matters for structured output prompts), and token count != word count. Good answers give a concrete example of a tokenization effect they've seen in practice.

Expect these follow-ups

• How do you estimate the token count of a prompt before sending it to the API?

As asked

When you compare query and document embeddings with cosine similarity, what are you actually measuring, and what does a similarity of 0.85 vs 0.60 mean in practice?

Sample answer outline

Cosine similarity measures the angle between two vectors, normalizing for magnitude. Values near 1 mean the vectors point in the same direction (semantically similar). 0.85 vs 0.60 depends entirely on the embedding model's distribution: you need to calibrate thresholds empirically for your specific model and domain. Good answers note that absolute values aren't interpretable without a distribution, and that cosine similarity is not always the right distance metric (inner product is sometimes preferred).

Expect these follow-ups

• When would you use Euclidean distance instead of cosine similarity for retrieval?

As asked

What is the difference between greedy decoding, beam search, and temperature sampling? For what kinds of tasks does each perform best?

Sample answer outline

Greedy: always picks the highest-probability next token; fast but repetitive and misses globally better sequences. Beam search: maintains k candidate sequences and picks the highest joint probability; better for translation and structured output but computationally heavier and still deterministic. Temperature sampling: samples from the distribution; better for creative tasks where diversity is valued. Most modern LLM APIs default to sampling and don't expose beam search.

Expect these follow-ups

• Why did beam search fall out of favor for open-ended generation tasks?

As asked

Explain how an approximate nearest neighbor index like HNSW or FAISS works, and why you'd use approximate rather than exact nearest neighbor search in a production RAG system.

Sample answer outline

HNSW (Hierarchical Navigable Small World) builds a layered graph where navigation starts at a high-level coarse layer and zooms into exact neighbors; FAISS uses quantization to compress vectors and search clusters. Both trade a small accuracy loss for massive speed gains. Exact k-NN over millions of vectors is O(n) per query; ANN indices are O(log n) or better. Good answers mention the recall/latency tradeoff and that the accuracy loss is usually acceptable for RAG because retrieval precision matters more than perfect completeness.

Expect these follow-ups

• How do you tune the recall vs. latency tradeoff in an HNSW index?

As asked

What is the difference between hallucination and confabulation in LLMs? How does grounding address these, and what types of errors does it not fix?

Sample answer outline

Hallucination in the broad sense covers any false output; confabulation specifically refers to the model generating plausible-sounding but invented content from its parametric memory. Grounding (providing external documents) addresses confabulation from memory but does not fix: the model misinterpreting or misquoting the grounded document, the model ignoring the document and relying on parametric knowledge, or an incorrect document being retrieved.

Expect these follow-ups

• What's harder to fix: the model ignoring good context, or the model being given bad context?

As asked

What is prompt injection, how does it differ from jailbreaking, and what makes it particularly dangerous in agentic systems?

Sample answer outline

Prompt injection: an attacker embeds instructions in user-controlled content (a document, a web page, a database record) that the model then executes as if they were trusted instructions. Jailbreaking: the user directly constructs a prompt to override the model's safety training. In agentic systems, prompt injection is especially dangerous because the agent takes real-world actions, so injected instructions can cause data exfiltration, unauthorized API calls, or lateral movement within a system.

Expect these follow-ups

• What's an example of an indirect prompt injection you've seen or can describe in detail?

As asked

In the context of a RAG retrieval step, what is the difference between precision and recall, and which matters more for the downstream LLM quality?

Sample answer outline

Precision: fraction of retrieved chunks that are actually relevant. Recall: fraction of relevant chunks that were retrieved. For RAG, recall is often more critical because if the right chunk is never retrieved the model cannot answer correctly regardless of generation quality. However, low precision fills the context window with noise, distracting the model. The candidate should note the tradeoff and that it depends on context window size and the model's ability to filter irrelevant context.

Expect these follow-ups

• Would you rather have precision 0.9/recall 0.5 or precision 0.6/recall 0.9? Justify for a specific task.

As asked

How would you use a natural language inference model to check whether a generated answer is faithful to its source document? What are the limits of this approach?

Sample answer outline

NLI models predict entailment, contradiction, or neutral for a premise-hypothesis pair. You use the source chunk as premise and the model's generated claim as hypothesis; contradiction or neutral signals a faithfulness issue. Limits: NLI models are trained on short texts and struggle with long premises, they can miss subtle factual errors (paraphrase that changes meaning slightly), and they don't handle numeric or temporal reasoning well.

Expect these follow-ups

• Would you use a small NLI model or an LLM judge for faithfulness at scale?

As asked

What is the difference between a hard (discrete) prompt and a soft (continuous) prompt? When would a researcher use soft prompt tuning instead of fine-tuning or hard prompt engineering?

Sample answer outline

Hard prompts are human-readable token sequences. Soft prompts are learned continuous vectors prepended to the input that are optimized by gradient descent without changing model weights. Soft prompt tuning is used when: full fine-tuning is too expensive, you need task-specific adaptation with minimal trainable parameters, or you want to keep the base model frozen for multi-task serving. Limit: soft prompts aren't human-interpretable and don't transfer across model architectures.

Expect these follow-ups

• If you moved to a new base model, what would happen to your soft prompts?