Generative model engineer machine learning interview questions

As asked

A product team wants a generative assistant to answer support questions in the company's tone using private documentation. When would you fine-tune, use RAG, or rely on prompting?

Sample answer outline

Use RAG when the main requirement is fresh private knowledge and citations, because fine-tuning is a poor way to memorise changing facts. Use fine-tuning when you need a consistent behaviour, format, style, or task policy that prompting cannot reliably maintain at scale. Prompting alone is suitable when the task is simple, low-risk, and changes frequently. Strong answers mention evaluation before and after each approach, plus the operational costs of data curation, retraining, retrieval quality, and latency. The common error is treating fine-tuning as a magic upgrade rather than a targeted behavioural intervention.

Expect these follow-ups

• What failure tells you the retriever is the problem?
• What data would you need for a useful fine-tune?
• How would you measure whether tone improved without making factuality worse?

As asked

A scientist proposes a new attention variant that improves long-context performance. How would you design the ablation study?

Sample answer outline

Define the hypothesis and isolate the moving part, otherwise the experiment becomes a comparison of entire stacks. Keep training data, token budget, optimiser, batch size, context length, and evaluation scripts constant unless the ablation explicitly tests one of them. Use tasks that reflect the claimed win, such as retrieval over long context, multi-hop reasoning, and latency or memory measurements at increasing sequence lengths. Strong candidates discuss confidence intervals, repeated seeds for noisy runs, and negative results that still teach something. The common mistake is celebrating one benchmark win while hiding compute cost or regressions on shorter-context tasks.

Expect these follow-ups

• How many seeds are enough for this experiment?
• What if the method wins only at a much higher compute budget?
• How do you report a promising but inconclusive result?

As asked

How would you build a synthetic data pipeline for instruction tuning without filling the training set with low-quality or duplicated examples?

Sample answer outline

Start with a target capability map so synthetic examples are generated for known gaps rather than volume for its own sake. Use strong prompts, multiple generators if possible, and filters for duplication, toxicity, format validity, factuality, and answer diversity. Human review should focus on calibration and hard categories rather than checking every example manually. Strong candidates discuss contamination, train-test leakage, model collapse from self-generated data, and weighting synthetic data below high-quality human data. The common trip-up is optimising token count instead of marginal training value.

Expect these follow-ups

• How do you detect near-duplicate examples at scale?
• When would synthetic data hurt model quality?
• How do you decide the mix of human and synthetic examples?

As asked

A team wants to ship an image generation feature for ecommerce product mockups. How would you evaluate the model before launch?

Sample answer outline

Define product-specific success criteria: prompt adherence, visual quality, brand safety, text rendering, object consistency, diversity, and editability. Use automated metrics only as weak signals, because FID-style metrics do not capture whether a generated product mockup is usable. Build human review rubrics and compare against realistic baselines such as stock imagery, templates, or a previous model. Strong answers include safety filters for protected brands, misleading product claims, and generated text in images. The trap is evaluating only aesthetic quality and missing commercial usability.

Expect these follow-ups

• How would you test consistency across a set of related product images?
• What categories should be blocked before launch?
• How would you measure whether users prefer the generated mockups?