As asked
A teammate is using plain k-fold cross-validation for everything. Explain when that gives you an honestly biased estimate, and what validation scheme you would use for time series and for grouped data.
Sample answer outline
Plain k-fold assumes rows are independent and identically distributed, and quietly lies when they are not. With time series it shuffles future and past together, so the model trains on the future to predict the past and the estimate is optimistic; use a forward-chaining scheme where each fold trains on the past and validates on the next slice, mirroring how the model will actually run. With grouped data, say multiple rows per customer, random folds put the same customer in both train and validation, leaking identity and inflating the score; use grouped folds that keep all of a group on one side. For classification with rare classes, stratify so each fold preserves the class balance and your estimate is stable. The point to land is that cross-validation is only honest when the splits respect the dependence structure of the data, so the validation setup must mimic the real prediction setting.
Expect these follow-ups
- Why does shuffled k-fold give an optimistic score on time series?
- What goes wrong when the same group lands in train and validation?
- When do you need stratified folds?