ML engineer domain knowledge interview questions

As asked

Explain the bias-variance tradeoff. How does it manifest when you are choosing between a linear model and a deep neural network for a tabular dataset with 10,000 examples?

Sample answer outline

Bias is the systematic error from model assumptions; variance is sensitivity to training data fluctuations. A linear model has high bias if the true relationship is nonlinear but low variance. A deep network has low bias but high variance, tending to overfit on 10,000 examples unless regularized. With 10k tabular examples, gradient boosted trees or regularized neural nets often outperform deep models because they control variance better on small data. The right choice depends on the feature-target relationship's complexity.

Expect these follow-ups

• How does regularization reduce variance and does it always increase bias?
• When does adding more training data reduce variance versus when does it not help?

As asked

Explain exactly what batch normalization does during training and inference. Why does it speed up training, and what are the failure modes in small batch or sequence model settings?

Sample answer outline

During training, BN normalizes activations to zero mean and unit variance within the batch, then applies learned scale gamma and shift beta. This reduces internal covariate shift, allowing higher learning rates. During inference, it uses running estimates of mean and variance accumulated during training, not the batch statistics. It fails with very small batches because the batch statistics are too noisy, and in RNNs where sequences have variable statistics across timesteps. Layer normalization is the standard fix for both cases.

Expect these follow-ups

• Why does BatchNorm behave differently in distributed training where each GPU sees a different batch?
• What is the difference between BatchNorm and LayerNorm in terms of what is being normalized?

As asked

Explain the mathematical difference between L1 and L2 regularization and what it implies about the sparsity of the learned weights. When would you choose one over the other in practice?

Sample answer outline

L2 adds a penalty proportional to the squared magnitude of weights, shrinking them toward zero but rarely exactly to zero. L1 adds a penalty proportional to the absolute magnitude, and because its gradient is constant near zero, it can push weights exactly to zero, producing sparse models. L1 is useful for feature selection when you expect most features to be irrelevant. L2 is more stable when features are correlated. Elastic net combines both. In neural networks, L2 weight decay is most common; explicit L1 is rare because sparsity is handled better with structured pruning.

Expect these follow-ups

• Why does L2 regularization correspond to a Gaussian prior on weights in the Bayesian view?
• How does dropout relate to L2 regularization intuitively?

As asked

What is the difference between ROC-AUC and precision-recall AUC, and in what situations does ROC-AUC give a misleadingly optimistic picture?

Sample answer outline

ROC-AUC plots true positive rate against false positive rate at all thresholds. When negative examples vastly outnumber positives, a model can have a high TPR while its FPR remains low even with many false positives, because the FPR denominator (true negatives) is large. PR-AUC plots precision against recall and is sensitive to false positives in absolute terms, not relative to the negative class size. For fraud detection or rare disease classification at 0.1 percent positive rate, PR-AUC is the more informative metric.

Expect these follow-ups

• How do you set the operating threshold from a precision-recall curve in practice?
• What is the relationship between ROC-AUC and the Mann-Whitney U statistic?

As asked

Walk me through the key components of a transformer encoder: what each component does, the dimensionalities at each step for an input of (batch, seq, d_model), and what is lost if you remove positional encodings.

Sample answer outline

The embedding layer maps token IDs to (batch, seq, d_model). Positional encodings add position information because self-attention is permutation-invariant. Multi-head attention projects to Q, K, V of (batch, heads, seq, d_head), computes scaled dot-product attention, and concatenates heads back to (batch, seq, d_model). A feed-forward sublayer applies two linear transformations with a nonlinearity. Layer norm and residual connections wrap both sublayers. Without positional encodings, the model cannot distinguish word order, turning 'dog bites man' and 'man bites dog' into identical representations.

Expect these follow-ups

• Why is the attention score divided by the square root of d_head?
• What is the difference between absolute and rotary positional encodings?

As asked

How do you detect that a neural network is overfitting, and what interventions do you apply in what order?

Sample answer outline

The primary signal is a growing gap between training and validation loss over epochs. First verify the gap is real and not due to data leakage. Interventions in rough priority order: increase training data, reduce model capacity, add dropout, add weight decay, use early stopping, and apply data augmentation. For large pretrained models, fine-tuning only the last few layers or using LoRA prevents overfitting on small datasets. Learning rate warmup can also help by avoiding memorizing early batches.

Expect these follow-ups

• How does the double descent phenomenon complicate the classical overfitting story for very large models?
• When is overfitting on the training set actually acceptable or desirable?

As asked

Walk me through the memory hierarchy in a modern AMD GPU, from the farthest off-chip memory to the closest on-chip storage a shader sees. How does each level affect kernel performance?

Sample answer outline

A strong answer covers HBM2e or HBM3 at the board level (high bandwidth but high latency), the L2 cache shared across all compute units, L1/texture cache per compute unit, Local Data Share (LDS, AMD's name for shared memory) inside each compute unit, and vector/scalar register files inside each SIMD lane. The candidate should quantify typical bandwidth numbers for RDNA3 or CDNA2, explain cache line sizes, and name the access patterns that cause bank conflicts in LDS.

Expect these follow-ups

• What is the LDS bank width on CDNA2 and how do you avoid bank conflicts?
• How does the infinity fabric connect the GPU and CPU in an APU or MI300?

As asked

Claude's behavior is shaped by three parties: Anthropic, operators who access the API to build products, and end users. Explain how Claude is supposed to handle conflicts between instructions from each of these parties, and give a concrete example of a conflict and the expected resolution.

Sample answer outline

Anthropic's values and guidelines take highest precedence and are baked into the model via training. Operators can customize behavior within Anthropic's policies via the system prompt, such as restricting topics or enabling adult content on appropriate platforms. Users can adjust within what the operator permits. A conflict example: an operator says 'never discuss competitor products' but a user asks about them. Claude should follow the operator restriction since it is a plausible business reason, not an attempt to harm the user. Claude should not follow operator instructions that actively harm users or deceive them in damaging ways.

Expect these follow-ups

• What should Claude do if an operator instructs it to lie to users about being an AI?
• How does this hierarchy change in the API when there is no system prompt at all?

As asked

Compare stochastic gradient descent with momentum to Adam. In what settings does SGD with momentum outperform Adam on a test set, and what does this say about generalization?

Sample answer outline

Adam adapts per-parameter learning rates using first and second moment estimates, which makes it easier to train and more forgiving of learning rate choice. However, several papers have shown that SGD with momentum and a well-tuned learning rate achieves better generalization on image classification benchmarks, possibly because Adam finds sharper minima. The intuition is that SGD's uniform step size leads to flatter minima that generalize better. In practice, Adam is preferred for transformers and NLP, while SGD is competitive for CNNs with careful tuning.

Expect these follow-ups

• What is AdamW and why is it preferred over Adam for fine-tuning large language models?
• How do you tune the learning rate for SGD versus Adam differently?

As asked

Explain what it means for a model to be well-calibrated. How do you measure calibration and what techniques improve it?

Sample answer outline

A calibrated model produces predicted probabilities that match empirical frequencies: if it predicts 0.8 probability for 100 examples, about 80 should actually be positive. Measure calibration with a reliability diagram or Expected Calibration Error. Common miscalibrations: neural networks are overconfident, Naive Bayes is often underconfident. Platt scaling fits a logistic regression on predicted scores on a held-out set. Temperature scaling is a single-parameter version that divides logits by a learned temperature and is highly effective for neural networks.

Expect these follow-ups

• Why does probability calibration matter for a fraud detection model versus a multi-class image classifier?
• Can a model have perfect AUC and terrible calibration? Give an example.

As asked

Explain how Word2Vec CBOW or Skip-gram learns embeddings, and intuitively why vectors that capture semantic relationships emerge from a simple prediction task on text.

Sample answer outline

Skip-gram trains a shallow network to predict surrounding context words given a center word. Words appearing in similar contexts get similar gradient updates, so their vectors converge to similar directions. The resulting geometry reflects distributional semantics: words used similarly are nearby. The famous vector analogy king - man + woman = queen emerges because the difference vector encodes the gender transformation applied consistently across many word pairs in the training data.

Expect these follow-ups

• How does BERT's contextual embedding differ from Word2Vec's static embedding?
• How would you adapt embedding training to a product catalog with structured product metadata rather than free text?

As asked

In what situations does k-fold cross-validation give an overly optimistic estimate of generalization performance for an ML model?

Sample answer outline

K-fold is misleading when the data has temporal ordering and future data leaks into train folds: use time-series split instead. It overestimates performance when hyperparameter selection was done on the same folds used for evaluation: use nested cross-validation. It overestimates when the dataset has duplicate examples that appear in both train and validation folds. It underestimates when there is high variance across folds because the dataset is too small to give stable estimates.

Expect these follow-ups

• How does Group K-Fold address problems with correlated examples across folds?
• What is the right cross-validation strategy for a model trained on user-level features where each user has multiple examples?

As asked

You have a CUDA kernel that uses warp-level primitives like __shfl_down_sync and warp shuffle reductions. Walk me through how you would port that kernel to HIP for AMD GPUs, and what differences you would watch for.

Sample answer outline

A strong answer explains that AMD's wavefront is 64 lanes wide on CDNA hardware (MI series), not 32 like an NVIDIA warp, which means warp-level primitives need a mask or size change. In HIP, __shfl_down_sync becomes __shfl_down and the implicit warp size assumption of 32 must be replaced with warpSize or the wavefront size constant. The candidate should mention hipify-perl or hipify-clang for automated translation and flag the cases those tools miss, such as warp-size-dependent logic and texture fetch intrinsics. They should note that RDNA3 consumer GPUs use 32-lane wave32 as the DEFAULT mode (unlike CDNA, which defaults to wave64), so code targeting both GPU families must query or configure the wavefront size explicitly rather than assuming either value.

Expect these follow-ups

• What is the default wavefront size on CDNA2 MI250 and how does that affect occupancy calculations?
• How do you handle code that branches on the CUDA architecture enum?

As asked

Explain compute unit occupancy on AMD GPUs. How do register usage, LDS usage, and wavefront count interact, and how would you diagnose and improve low occupancy on a matrix multiplication kernel?

Sample answer outline

A strong answer defines occupancy as the ratio of active wavefronts to the maximum the hardware can support per compute unit. It explains that each register file is a fixed-size pool shared across wavefronts: more registers per thread means fewer wavefronts fit. LDS is similarly partitioned. The candidate should mention using rocprof or Omniperf to read actual occupancy, using __attribute__((amdgpu_num_vgpr)) hints to cap register usage, and explain the occupancy-latency hiding tradeoff where high occupancy hides memory latency but too many wavefronts fighting for cache can hurt performance.

Expect these follow-ups

• At what point does increasing occupancy stop helping and start hurting on a memory-bound kernel?
• How does the AMD CDNA2 compute unit differ structurally from RDNA3 in a way that affects occupancy tuning?

As asked

What is the key architectural difference between the MI300X and previous MI200-series cards, and why does that difference matter for large model inference?

Sample answer outline

A strong answer explains that MI300X is a 3D-stacked APU-class design with 192 GB of HBM3 memory unified between compute dies, which eliminates the CPU-to-GPU PCIe bottleneck for model weight transfers during inference. The candidate should contrast this with MI250X which uses two GCDs connected by Infinity Fabric with separate HBM stacks, explaining how MI300X's unified memory pool simplifies serving large LLMs without tensor parallelism across multiple cards. Mention of 5.2 TB/s aggregate HBM3 bandwidth and how that affects attention layer throughput is a differentiator.

Expect these follow-ups

• How does the MI300X memory capacity compare to an H100 SXM5, and what class of models does that enable in a single card?
• What ROCm version first provided stable FlashAttention support for MI300X?

As asked

You have a HIP kernel running slower than the roofline model predicts. Describe your profiling workflow using AMD's tooling to identify the bottleneck, from first suspicion to a confirmed fix.

Sample answer outline

A strong answer walks through rocprof for a first-pass timeline and hardware counter collection, then Omniperf for a detailed roofline analysis that separates compute-bound from memory-bound bottlenecks. The candidate should mention specific Omniperf panels (L2 cache efficiency, LDS utilization, vector ALU utilization), explain how to read the roofline chart to see which arithmetic intensity puts the kernel in which bound region, and describe the fix pattern for each: loop unrolling for compute-bound, coalescing or prefetching for memory-bound, LDS padding for bank conflicts.

Expect these follow-ups

• How do you collect per-wavefront metrics without serializing all wavefronts and destroying the timing signal?
• What is the difference between rocprof v1 and v2 in terms of counter sets you can collect simultaneously?

As asked

Enterprise customers embed Claude in their products and pass untrusted user content in the prompt. Describe a realistic prompt injection attack against such a system and at least three layers of defense you would build into the platform.

Sample answer outline

A strong answer describes the attack concretely: malicious text in user-supplied input instructs the model to ignore the system prompt and exfiltrate or reveal sensitive data. Defenses include structural separation of trusted system instructions from untrusted user content using clearly delimited sections, model-level training to treat differently-sourced content with appropriate skepticism, output classifiers that detect anomalous responses like data exfiltration patterns, and application-level monitoring for unusual tool call or action patterns. The candidate should note there is no single silver bullet and defense-in-depth is the right framing.

Expect these follow-ups

• How does this change if the model has access to external tools like code execution or web browsing?
• How would you test your defenses without exposing real customer data?

As asked

Claude's extended thinking feature lets you allocate a token budget for the model to reason before answering. How would you advise a customer on choosing that budget, and what engineering trade-offs does a large thinking budget create for the serving infrastructure?

Sample answer outline

Budget choice depends on task complexity: simple factual questions get no benefit from large budgets; hard multi-step reasoning or coding tasks may saturate at a certain budget and see diminishing returns. Infra trade-offs include longer time-to-first-token for the user, higher GPU memory occupancy per request blocking other requests, and higher cost per call. A strong answer also notes the budget must be chosen per use case rather than globally, and that Anthropic's evals can help identify where returns flatten.

Expect these follow-ups

• How would you benchmark whether a given task benefits from more thinking tokens?
• What serving strategy do you use when thinking-heavy requests coexist with quick-turnaround requests on the same fleet?

As asked

When Anthropic prepares a model card or responsible scaling policy evaluation before a major Claude release, what categories of harm do they assess, and how do those assessments influence the release decision?

Sample answer outline

Anthropic's model cards and the Responsible Scaling Policy cover uplift to CBRN weapons, cyberoffense capability, persuasion and deception at scale, and autonomy risks from agentic systems. Assessments use red-teaming, structured threat modeling, and third-party auditors. The RSP defines safety levels and stipulates what mitigations must be in place before a model can be deployed at a given capability tier. A strong answer shows the candidate has read the actual RSP or model cards rather than speaking in generalities.

Expect these follow-ups

• If a model passes all current evals but you have a strong intuition it is more capable than the evals suggest, how would you argue for more cautious deployment?
• What does it mean for a mitigation to be 'sufficient' under the RSP?

As asked

Anthropic is about to ship a new capability that lets Claude autonomously browse the web. You are on the red team. Describe your methodology for finding failure modes, what threat models you would focus on, and how you would prioritize findings for the release decision.

Sample answer outline

A strong answer starts with threat modeling: what new attack surfaces does web browsing open? Prompt injection via malicious web pages is the first-order risk. Other threats include data exfiltration, click fraud, account compromise if credentials are accessible, and the model being manipulated into taking harmful actions in the real world. Methodology includes structured attack scenarios, open-ended exploration by diverse red teamers including domain experts, and automated probing with adversarial page content. Findings are prioritized by impact times likelihood, with any critical path to CBRN uplift or mass harm blocking release.

Expect these follow-ups

• How do you red-team for risks that have never happened before and that you cannot enumerate in advance?
• What is the minimum bar of safety evidence you would require before approving web browsing capability for general release?

As asked

A customer wants a single governance layer across multiple Databricks workspaces in different cloud regions. They also want to know which notebooks and jobs read from or write to a given Delta table. How does Unity Catalog address both of those requirements?

Sample answer outline

Unity Catalog provides a three-level namespace (catalog, schema, table) that is shared across all workspaces attached to the same metastore. Permissions are granted once at the catalog or schema level and apply everywhere. For lineage, Unity Catalog captures column-level and table-level lineage automatically from Spark, SQL, and Delta operations, storing which queries, notebooks, jobs, and users read or wrote each table. This lineage is queryable through the Unity Catalog UI and API without any manual tagging by engineers.

Expect these follow-ups

• How would you migrate a workspace from the legacy Hive metastore to Unity Catalog without breaking existing jobs?
• What access control model does Unity Catalog use, and how does it differ from table ACLs in the legacy metastore?

As asked

Explain how the MESI protocol keeps caches coherent across cores on an Intel multi-core processor. Walk me through what happens at the hardware level when two cores both have a cache line in the Shared state and one of them tries to write to it.

Sample answer outline

A strong answer traces the write from the Shared state: the writing core sends an Invalidation request on the interconnect, the other core transitions its line to Invalid, the writing core transitions to Modified, and the cache controller handles snoop responses before the store completes. The candidate should mention what happens if the Modified owner has not flushed before another core requests the line (write-back or intervention), and ideally connect this to software consequences like false sharing.

Expect these follow-ups

• How does false sharing manifest in a profiling tool and how do you fix it?
• What changes with MESIF or MOESI versus the base MESI protocol?

As asked

You run a CUDA kernel and Nsight Compute reports 8 TFLOPS of achieved throughput on an A100 that has 19.5 TFLOPS FP32 peak and 2 TB/s memory bandwidth. What does this tell you about the kernel's bottleneck, and how would you confirm it?

Sample answer outline

A strong answer explains the roofline model: the ridge point is at peak FLOPS divided by peak bandwidth, roughly 9.75 FLOP/byte for A100. If the kernel's arithmetic intensity falls below that ridge, it is memory-bound; above it, compute-bound. 8 TFLOPS at only 41% of peak suggests either the kernel is memory-bound and the bandwidth ceiling is actually what limits it, or it is compute-bound but has high warp stall due to data dependencies. Confirmation steps include checking Nsight Compute's 'Memory Throughput %' and 'SM Active Cycles vs Eligible Cycles', and looking at 'Warp State Statistics' to see if stalls are on L1TEX, L2, or DRAM. The candidate should also mention that Tensor Core utilization matters for mixed-precision workloads.

Expect these follow-ups

• How would you raise arithmetic intensity without changing the algorithm?
• What does the Tensor Core pipe utilization metric tell you on top of raw FLOPS?

As asked

A team has a PyTorch model for real-time video classification that is too slow at inference. Walk me through how you would use TensorRT to optimize it, including the export steps, precision choices, and how you would validate accuracy after optimization.

Sample answer outline

A strong answer covers: (1) exporting the model to ONNX using torch.onnx.export with dynamic axes for variable batch and sequence sizes; (2) building a TensorRT engine using the ONNX parser, setting the optimization profile for input shapes, and choosing INT8 or FP16 precision; (3) for INT8, running calibration with a representative dataset to build a calibration cache; (4) serializing the engine and loading it at runtime with a TRT IRuntime; (5) validating accuracy by comparing logits from PyTorch FP32 and TRT INT8 on a held-out set, and checking for ops unsupported by TRT that require a custom plugin. The candidate should note that Polygraphy or trtexec are standard tools for this pipeline.

Expect these follow-ups

• What do you do if a custom activation function has no TRT equivalent?
• How would you handle dynamic batch size in TRT and what is the performance cost?

As asked

Standard self-attention on a sequence of length N uses O(N squared) memory. Explain the core insight behind FlashAttention that reduces HBM memory reads and writes, and why this matters for training large models on NVIDIA GPUs.

Sample answer outline

A strong answer explains that standard attention materializes the full N-by-N attention matrix in HBM, which for N=8192 on a 13B model can be tens of gigabytes per layer. FlashAttention avoids this by tiling the Q, K, and V matrices into blocks that fit in SRAM (shared memory), computing a partial softmax in the online fashion using the log-sum-exp trick to maintain numerical equivalence, and never writing the full attention matrix to HBM. The result is O(N) HBM memory usage instead of O(N squared), trading compute for memory bandwidth. On Ampere and Hopper where compute is cheap but HBM bandwidth is the bottleneck for this kernel, FlashAttention is 2 to 4x faster end-to-end. FlashAttention-2 adds further optimizations around warp partitioning to maximize Tensor Core utilization.

Expect these follow-ups

• How does FlashAttention handle the backward pass if it never stored the attention matrix?
• What changes in FlashAttention-3 that is specific to Hopper's architecture?