As asked
Your usability tests suggest users hate a new onboarding step, but analytics shows users who complete it retain better. How do you make a recommendation?
Sample answer outline
Treat the evidence as answering different questions: usability tests reveal friction and comprehension, while analytics may be affected by selection bias because only motivated users complete the step. Segment the data, check whether the step causes retention or merely filters for intent, and inspect where users drop. The recommendation might be to preserve the valuable commitment or education while redesigning the friction. A senior answer does not pick the evidence source they prefer; it names the uncertainty and proposes the next decision-quality test.
Expect these follow-ups
- How would you test whether the onboarding step causes retention?
- What would you tell a PM who wants to remove it immediately?
- How do you present uncertainty without sounding indecisive?
synthesisanalyticsdecision-making