Scenario & System Design for ML
Open-ended design questions — CCTV counting, fraud detection, model diagnosis.
Start here
The hardest module to prep by memorization — there's no fixed answer, just a vague real problem ("count the cars in this video") graded on how you think, not what you blurt first.
The trap: jumping straight to a technique ("I'd use a !") before understanding the problem. The fix — work through a fixed sequence out loud before naming any technique: What does success mean, in business terms? What data exists, what's missing? What's the simplest , and why would more complexity be worth it? How will we measure whether it worked? What constraints matter (real-time, explainable, skewed data)? What happens when it's wrong in production?
Walk that sequence out loud and the right technique usually becomes obvious — a panel that watches you reason like this trusts you far more than a candidate who guesses an architecture name in ten seconds.
Why this module matters
This is where Therap-style and bKash-style boards differentiate strong candidates — there's no single correct answer, they're grading your *reasoning process*.
A reusable framework (say this structure out loud)
1. Clarify the goal — what does success look like? What's the business metric behind the ML metric?
2. Data — what's available, what's missing, labeled or not, how much, how skewed?
3. Approach — first (simple, explainable), then justify complexity added.
4. Evaluation — the *right* metric for this problem (not just ).
5. Constraints — , , interpretability, cost, fairness.
6. Failure modes — what breaks this in production, how would you monitor it?
Worked archetypes to rehearse
- Count distinct cars from CCTV given a per-frame detector → this is a *tracking* problem, not just detection: assign IDs across frames using matching or a tracker (SORT/DeepSORT-style), count unique track IDs, handle occlusion/re-entry.
- Classify unlabeled news articles at high → no labels means start with /keyword rules or , use that to bootstrap a labeled set, then distill into a small fast for production .
- Diagnose a from train/val loss curves → both high & flat = (add capacity/features); train low, val high & diverging = (regularize/more data); both dropping but val noisy/spiking = too high or size too small.
- Fraud detection for a mobile wallet (bKash-relevant) → severe (fix: resampling/class /anomaly detection framing), need high on fraud without drowning ops in false positives (- tradeoff), must score in near-real-time ( budget), ideas: transaction velocity, device/location change, amount deviation from user's history.
- Credit scoring for thin-file/unbanked users (bKash-relevant) → no bureau history, so wallet/telco behavioral data becomes the signal; explainability and fairness are hard constraints (lending is regulated); handle with staged limits, and watch reject-inference — you only observe repayment for the users you approved.
Viva angle
Never blurt an architecture name first — walk the framework, then land on a technique. The reasoning trail is the answer.
Visual reference
Multi-object tracking across frames
Data drift vs concept drift
Cheatsheet
The 6-step framework
| # | Ask |
|---|---|
| 1 | Goal — what does success mean, in business terms? |
| 2 | Data — what exists, what's missing, how skewed? |
| 3 | Approach — simplest baseline first, justify complexity |
| 4 | Evaluation — the right metric, not just accuracy |
| 5 | Constraints — latency, cost, interpretability, fairness |
| 6 | Failure modes — what breaks, how would you notice? |
Loss curve diagnosis
| Pattern | Diagnosis |
|---|---|
| Both high & flat | Underfitting |
| Train low, val high & diverging | Overfitting |
| Both dropping, val noisy/spiking | Learning rate too high |
| Both dropping, painfully slow | Learning rate too low |
- ▸Say the framework out loud before naming any technique.
- ▸"Count objects in video" is a tracking problem ( + track IDs), not just detection.
- ▸No labels yet? / zero-shot → bootstrap labels → distill to a small fast .
- ▸Lending models must be explainable — a regulator can demand the reason for every rejection.
- ▸Never hard-swap a that only won offline: shadow → canary → ramp, with rollback ready.
Further study
Question bank
15 questionsYou have a per-frame object detector for cars. Design a system to count the number of *distinct* cars seen in a CCTV video (not just detections per frame).
You need to classify an incoming stream of unlabeled news articles into 3 categories at high , with no labeled training data available yet. How would you approach this?
Given a plot of train and validation loss over epochs, walk through how you diagnose whether the is , , or has a learning-rate problem.
Design a fraud detection system for a mobile financial service wallet. Cover features, , evaluation, and .
What general framework/structure should you follow when answering any open-ended ML system design question in a viva?
How would you design a personalized offer/recommendation system for a mobile wallet's promotions?
When would you use prediction versus real-time/online serving for a in production? Give concrete examples.
What's the difference between data drift and concept drift, and how would you detect each in a production ML system?
What problem does a store solve, and why would an ML platform team invest in building one?
Why is reproducibility in ML more than just saving ? What else needs to be versioned?
How does CI/CD for ML differ from CI/CD for regular software?
Why does explainability (e.g. SHAP/LIME) matter specifically for a regulated fintech context like credit or loan decisions?
What practical privacy and PII concerns come up when training ML models on financial transaction data, and how do you handle them?
Design a credit scoring for mobile wallet users with no formal credit history ("thin-file"/unbanked users). What data, choice, and constraints matter?
Your new fraud beats the current one offline. How do you roll it out to production safely?