Data scientist technical fundamentals interview questions

As asked

Explain intuitively, without relying on equations, why adding a penalty term to the loss function prevents a model from memorizing the training data.

Sample answer outline

Regularization penalizes model complexity, forcing the model to 'earn' each increase in weight by a corresponding improvement in training loss. Without it, the model can freely grow weights to fit even noise in the training data. With a penalty, the optimizer will only make weights large when the signal in the data is strong enough to outweigh the penalty cost, so the model learns the true pattern rather than memorizing individual examples.

Expect these follow-ups

• How does dropout regularization in neural networks differ mechanistically from L2 regularization?

As asked

You are choosing between Random Forest and XGBoost for a tabular classification problem. Walk me through the factors that would make you choose one over the other.

Sample answer outline

Random Forest trains trees in parallel (faster for large datasets), is more robust to hyperparameter settings, handles missing values less gracefully (requires imputation), and is harder to overfit. XGBoost typically achieves higher accuracy on structured data through sequential error correction, handles sparse data and missing values natively, but is more sensitive to hyperparameter tuning and slower to train. Choose Random Forest when training time or tuning budget is limited; choose XGBoost when maximizing predictive performance with sufficient tuning time.

Expect these follow-ups

• What is the difference between bagging and boosting in terms of the bias-variance tradeoff?

As asked

A colleague says you cannot use a t-test because your revenue metric is heavily right-skewed. How do you respond, and what sample size would make you comfortable proceeding?

Sample answer outline

The Central Limit Theorem guarantees that the sampling distribution of the mean approaches normality as sample size grows, regardless of the underlying distribution. For moderately skewed distributions, n=30 is often cited as a rule of thumb, but for heavy-tailed metrics like revenue, n=200 or more may be needed. The candidate should mention that they would check with a bootstrap distribution of the mean to verify normality, and consider log-transforming or using a non-parametric test like Mann-Whitney U as an alternative.

Expect these follow-ups

• When would you use a Mann-Whitney U test instead of a t-test, and what does it actually test?

As asked

A test for a disease has a 99% sensitivity and 95% specificity. The disease affects 1 in 10,000 people. You test positive. What is the probability you actually have the disease?

Sample answer outline

Using Bayes theorem: P(disease | positive) = P(positive | disease) * P(disease) / P(positive). P(positive) = P(positive | disease) * P(disease) + P(positive | no disease) * P(no disease) = 0.99 * 0.0001 + 0.05 * 0.9999 = 0.000099 + 0.049995 = 0.050094. P(disease | positive) = 0.000099 / 0.050094, which is approximately 0.197%, or roughly 1 in 500. This illustrates base rate neglect. The candidate should walk through this calculation clearly and explain why a highly accurate test can still be mostly wrong when the base rate is very low.

Expect these follow-ups

• How does this reasoning apply when you are interpreting p-values from a large multiple-testing study?

As asked

In a SQL query that has both a GROUP BY clause and a window function, in what order do they execute, and what does that imply about which values the window function can see?

Sample answer outline

SQL logical execution order: FROM, WHERE, GROUP BY, HAVING, SELECT (window functions evaluated here, after aggregation), ORDER BY. This means a window function in the SELECT clause sees only the grouped rows, not the original pre-aggregated rows. If you want to apply a window function to unaggregated data, it must be done in a subquery or CTE before the GROUP BY. The candidate should illustrate with an example where a running total over monthly grouped rows behaves differently from one over the raw event rows.

Expect these follow-ups

• Why can you not filter on a window function result in a WHERE clause without wrapping it in a CTE or subquery?

As asked

How would you use expected value to decide whether to ship a feature that showed a 2% lift in revenue but a 1% lift in support ticket volume? Walk through the calculation.

Sample answer outline

Expected value requires quantifying the dollar value of a 2% revenue lift and the cost of a 1% increase in support tickets. The candidate should walk through a concrete example: if annual revenue is 10M, a 2% lift is 200k. If each support ticket costs 15 dollars and volume is 100k tickets per year, a 1% increase is 1,000 tickets or 15k dollars. Net expected value is 200k minus 15k = 185k, ship it. The candidate should also discuss uncertainty around both estimates and the importance of using confidence intervals rather than point estimates.

Expect these follow-ups

• How does risk tolerance affect whether you ship when the expected value is positive but uncertain?

As asked

Explain how MapReduce works conceptually, then write the map and reduce functions to count word frequencies across a large corpus of documents that does not fit in memory on a single machine.

Sample answer outline

A strong answer explains that the map phase produces (key, value) pairs from input chunks in parallel across many workers, and the reduce phase aggregates values for each unique key after a shuffle and sort step. For word count, map emits (word, 1) per word and reduce sums the values for each word. The candidate should mention that the shuffle is the bottleneck, explain how combiners reduce network traffic, and note that MapReduce is largely superseded inside Google by FlumeJava (the inspiration for Apache Beam and Cloud Dataflow), but the concept remains foundational and still appears in interview discussions.

Expect these follow-ups

• What is a combiner and when does it help?
• How would you handle a skewed key distribution where one word appears in 90% of documents?

As asked

Snowflake uses columnar storage optimized for analytics. Explain exactly why columnar storage is faster for aggregate queries but worse for point lookups, how data is compressed in columnar format, and when you would still reach for a traditional row store.

Sample answer outline

Columnar storage keeps all values of a single column contiguous on disk, so a SUM(revenue) query reads only the revenue column, skipping all other columns. This dramatically reduces I/O for wide tables. Compression is much more effective per column because values in a column share the same type and often have low cardinality (run-length encoding, dictionary encoding, delta encoding all apply well). For a point lookup by primary key (fetch one row), you must read from each column file to reconstruct the row, making it slower than a row store where one disk seek gets the full row. Row stores excel at high-concurrency OLTP with frequent single-row reads and writes.

Expect these follow-ups

• How does Snowflake store semi-structured data (VARIANT columns) differently from typed columns?
• If a query needs all 50 columns of a table, does columnar storage still help? Why or why not?

As asked

Your model shows that income strongly predicts health outcomes, but income is correlated with education. Is education a confounder or a mediator here, and why does the distinction matter for your analysis?

Sample answer outline

Education is a confounder if it is a common cause of both income and health but is not on the causal path from income to health. It is a mediator if income causes higher education which in turn causes better health, so income affects health partly through education. The distinction matters because conditioning on a mediator blocks the very causal path you are trying to estimate, producing what is sometimes called over-adjustment bias. Conditioning on a confounder, by contrast, is necessary to remove bias. Mistaking a mediator for a confounder and controlling for it will underestimate the total effect of income on health. The candidate should reference a directed acyclic graph (DAG) as the right tool for making this determination.

Expect these follow-ups

• How would you use a DAG to determine whether to include a variable in a regression model?

As asked

Early neural networks used sigmoid activations but modern deep networks use ReLU and its variants. Explain why this change happened and when you would still use sigmoid.

Sample answer outline

Sigmoid saturates at both ends, causing vanishing gradients in deep networks (gradient approaches zero, weights stop updating). ReLU has a gradient of 1 for positive inputs, solving the vanishing gradient problem for deep networks. However, ReLU has the dying ReLU problem (units stuck at zero for all inputs). Leaky ReLU and ELU fix this. Sigmoid is still used in the output layer for binary classification and in gating mechanisms (LSTM). The candidate should mention that the computational cost of ReLU is also lower.

Expect these follow-ups

• What is the dying ReLU problem and how does Leaky ReLU address it?

As asked

Why does the curse of dimensionality make k-nearest neighbors unreliable in high-dimensional spaces, and what does this imply about feature engineering?

Sample answer outline

In high dimensions, all points become approximately equidistant from each other because most volume is near the surface of the hypersphere. This means nearest neighbors are no longer meaningfully close, so k-NN predictions degrade. For feature engineering, this implies that adding irrelevant dimensions hurts distance-based models significantly, so dimensionality reduction (PCA, feature selection) is critical before applying k-NN or density-based clustering. It also explains why deep learning models that learn low-dimensional representations outperform naive methods on raw high-dimensional inputs.

Expect these follow-ups

• How does this problem affect kernel SVM in high dimensions?

As asked

Explain what the attention mechanism in a transformer does in plain terms, without using the words 'query', 'key', and 'value' as jargon. Why is this better than the fixed-length context vector in RNNs?

Sample answer outline

Attention lets each output token look back at all input tokens and decide which ones are most relevant to it, producing a weighted combination of input representations. This is better than RNNs which compress all input information into a single fixed-size vector and then try to decode from it, which loses information for long sequences. The candidate should explain that attention is O(n^2) in sequence length, which is why it became computationally feasible only as hardware scaled, and why sparse attention mechanisms exist.

Expect these follow-ups

• What is the computational complexity of self-attention with respect to sequence length, and what architectural changes have been proposed to reduce it?