ML interviews are failed on depth, not knowledge.

Practise realistic ML system design questions, as a back-and-forth with follow-ups or in writing. Every answer comes back marked criterion by criterion, so you know where you'd lose points.

Start free See a marked answer

Recommendation systems, staff level

Design a real-time recommendation system for a video platform with 50 million daily active users.

17/25

Pick a criterion to see which part of the answer earned or lost it.

Candidate answer, excerpt

I'd start with the objective. I'd optimise for expected watch time rather than clicks, since CTR rewards thumbnails, not satisfaction. At 50 million daily users and around 20 feed loads each, that's roughly a billion requests a day, so every stage needs a latency budget. I'll assume this is the home feed on mobile.

For retrieval, a two-tower model with user and video embeddings, served through approximate nearest neighbour search, pulling about a thousand candidates in under 50 ms. I'd add a second source for trending videos so the feed isn't purely personalised.

The ranker is a gradient-boosted model over watch history, freshness, creator features and device context, trained on logged impressions with watch time as the label. Clicks on the first row go straight into training. A lightweight re-ranker then enforces diversity so one creator can't fill the screen.

To evaluate, I'd track NDCG and watch time and ship if both go up. The A/B test would run for two weeks to cover weekly patterns.

For new users we'd probably just show popular videos. No plan for new videos.

Problem framing3/5
Right objective and a scale estimate, but assumed the surface instead of asking.
Next question Before you draw anything, what are we optimising for, and for which users?
Candidate generation5/5
Justified with a latency budget, not recited.
Next question Why two-tower instead of matrix factorisation?
Ranking4/5
Strong features, watch time as the label and a diversity re-ranker. Loses a point for position bias: top-row clicks are mostly position, not relevance.
Next question What is the ranker actually learning from those clicks?
Evaluation4/5
Offline and online metrics blurred together.
Next question NDCG went up offline but watch time dropped. Why?
Cold start1/5
Mentioned in passing. Nothing for new videos. A staff-level answer raises it unprompted.
Next question A user signed up ten minutes ago. What's in their feed?

Passes at senior. Not yet at staff.

Answer this question free

Recommendation systems, staff level

Design a real-time recommendation system for a video platform with 50 million daily active users.

17/25

I'd optimise for expected watch time rather than clicks, since CTR rewards thumbnails, not satisfaction. At 50 million daily users and around 20 feed loads each, that's roughly a billion requests a day, so every stage needs a latency budget. I'll assume this is the home feed on mobile.
Right objective and a scale estimate, but assumed the surface instead of asking.
Next question Before you draw anything, what are we optimising for, and for which users?
For retrieval, a two-tower model with user and video embeddings, served through approximate nearest neighbour search, pulling about a thousand candidates in under 50 ms. I'd add a second source for trending videos so the feed isn't purely personalised.
Justified with a latency budget, not recited.
Next question Why two-tower instead of matrix factorisation?
The ranker is a gradient-boosted model over watch history, freshness, creator features and device context, trained on logged impressions with watch time as the label. Clicks on the first row go straight into training. A lightweight re-ranker then enforces diversity so one creator can't fill the screen.
Strong features, watch time as the label and a diversity re-ranker. Loses a point for position bias: top-row clicks are mostly position, not relevance.
Next question What is the ranker actually learning from those clicks?
For new users we'd probably just show popular videos. No plan for new videos.
Mentioned in passing. Nothing for new videos. A staff-level answer raises it unprompted.
Next question A user signed up ten minutes ago. What's in their feed?
To evaluate, I'd track NDCG and watch time and ship if both go up. The A/B test would run for two weeks to cover weekly patterns.
Offline and online metrics blurred together.
Next question NDCG went up offline but watch time dropped. Why?

To evaluate, I'd track NDCG and watch time and ship if both go up. The A/B test would run for two weeks to cover weekly patterns.

For new users we'd probably just show popular videos. No plan for new videos.

Passes at senior. Not yet at staff.

Answer this question free

See exactly where the points went

Most practice tells you whether you were right. ModelCandidate scores each criterion separately and tells you what earned marks, what cost them and what to fix. In conversation mode, the interviewer pushes on your weak spots as you go.

Freeto start. No card needed.

Prefer to talk it through?

Conversation mode is a realistic back-and-forth. The interviewer follows up on what you said, changes the constraints, and pushes on the weak spots. You're marked on the same rubric.A realistic back-and-forth. The interviewer follows up on what you said and pushes on the weak spots. Same rubric.

Try a mock interview

Mock round, recommendation systems, 14 minutes in

Interviewer: You said a thousand candidates in under 50 ms. What happens if product wants five thousand?
You: Retrieval stays cheap because it's ANN. The heavy ranker becomes the bottleneck, so I'd add a light first-pass ranker to cut five thousand down to five hundred.Ranking +
Interviewer: Most of your training clicks come from the first row. What is the ranker actually learning?
You: Mostly what users like. We could add more negatives to balance it out.Position bias −
Interviewer: A user signed up ten minutes ago. What does their feed show?
You: Popular videos, I suppose.Cold start −

Same rubric, same scorecard at the end. The difference is you have to think on your feet.Same scorecard at the end. The difference is you have to think on your feet.

Try a mock interview

Try a few quiz questions

Eight sample questions. Pick an answer to see why each option is right or wrong.Eight sample questions. Pick an answer to see why it's right or wrong.

NLP

Question 1 of 8

A decoder predicting the next token must not attend to future tokens. Which mechanism enforces this?

Pick an answer.

Built for the ML round, not the coding round

900+ quiz questions across 12 domains. Mock interviews cover ML system design, recommendation systems, ranking, NLP and computer vision.

Mock interviews

ML System Design, Recommendation Systems, Ranking, NLP, Computer Vision.

The cheapest part of changing jobs

Free

Selected interview questions, both modes
Marked feedback on each one
15 quiz answers every 24 hours

Start free

Pro

$29 / month

Billed monthly. Cancel any time. Includes any applicable tax.

The full interview question bank
Unlimited quizzes, by difficulty and company style
Marked feedback on every answer

Start free, upgrade later

Find out where you'd lose points before the real loop does.

Start free