ML Fundamentals
Supervised/unsupervised, bias-variance, regularization, evaluation metrics.
Start here
A normal program follows rules a human wrote by hand. ML flips that: you show the computer thousands of examples and it works out the rules itself — that's called training, and the result is a .
- — a measurable input (house size, transaction amount).
- — the known correct answer for an example (actual sale price).
- — learn from (, ) pairs to predict labels on new data. Most real-world ML starts here.
- — find structure with no labels at all (e.g. group similar customers).
- — an agent learns from reward/penalty via trial and error, no fixed dataset.
Once a exists, everything below boils down to two questions: did it learn the real pattern or just memorize the training examples, and how do we honestly measure whether it's good? /, , and evaluation metrics are the detailed answers.
Why this module matters
This is the "rapid-fire" core of any ML viva — panels open here because it tests whether you *understand* concepts, not just recite library calls.
Core threads
- Learning paradigms: supervised ( → predict), unsupervised (find structure, no labels), reinforcement (agent + reward signal, learns via trial and error).
- – tradeoff: = error from overly simple assumptions (); = error from being too sensitive to training data (). Total error = ² + + irreducible noise.
- : L1 () shrinks some to exactly zero → selection; L2 () shrinks smoothly → handles multicollinearity. Both fight by penalizing large .
- Evaluation: lies on imbalanced data (e.g. 1% fraud — predicting "no fraud" always gives 99% ). Use , , F1, , PR-AUC instead.
- Validation discipline: train/val/test split, , and the cardinal sin — (test information sneaking into training, e.g. scaling before splitting).
Viva angle
Expect: "explain X in one minute", "when would you prefer L1 over L2", "your has 99% but the recruiter is unhappy — why?" Practice saying these out loud, not just recognizing them on a flashcard.
Visual reference
Bias–variance tradeoff
Confusion matrix
L1 vs L2 constraint regions
Cheatsheet
Accuracy
(TP + TN) / Total
Misleading on imbalanced data.
Precision
TP / (TP + FP)
Of flagged positives, how many were right.
Recall
TP / (TP + FN)
Of actual positives, how many were caught.
F1 score
2·(P·R) / (P + R)
Harmonic mean of precision & recall.
Total error
Bias² + Variance + Noise
The bias-variance decomposition.
Confusion matrix
| Predicted + | Predicted − | |
|---|---|---|
| Actual + | TP | FN |
| Actual − | FP | TN |
L1 vs L2 regularization
| L1 (Lasso) | L2 (Ridge) | |
|---|---|---|
| Penalty | Σ|w| | Σw² |
| Effect | Some weights → exactly 0 | All weights shrink smoothly |
| Good for | Feature selection | Multicollinearity |
Bagging vs boosting
| Bagging | Boosting | |
|---|---|---|
| Training | Parallel, independent | Sequential, each fixes last |
| Reduces | Variance | Bias |
| Example | Random Forest | Gradient Boosting |
- ▸Underfit = high = poor on train AND val.
- ▸Overfit = high = great on train, poor on val.
- ▸ CV: average the score across k train/val splits.
- ▸Fit preprocessing (scalers etc.) on train fold only — never on the full dataset.
Further study
Question bank
27 questionsWhat is the difference between supervised, unsupervised, and ? Give one example of each.
Explain the – tradeoff. How do you detect and fix vs ?
L1 vs — how are they different geometrically, and when would you prefer one over the other?
Your fraud-detection has 99% but the fraud team says it's useless. Why, and what metric should you use instead?
Define , , and . What does the balance?
What is ? Give an example of how it can silently happen during preprocessing.
Why do we need a separate validation set in addition to train and test?
Compare decision trees, , and . What is the core difference between and ?
Why do we use in instead of just fitting a line and thresholding?
Explain clustering and one weakness of the algorithm.
What is the "" in , at an intuition level?
What does do, intuitively, and what is it typically used for?
What is the difference between , stochastic, and ?
How does act as ?
What is and why is it more reliable than a single train/val split?
You have severely imbalanced data (1% positive class). Name three distinct strategies to handle this, beyond just changing the metric.
Compare grid search, random search, and Bayesian optimization for tuning. Which would you use in practice?
What is cross-entropy (log loss), and why do we use it instead of MSE for classification?
What is Maximum Likelihood Estimation, intuitively, and how does it connect to the loss functions we use in training?
What does it mean for a 's predicted probabilities to be "calibrated"? Why does this matter for something like a fraud risk score?
ROC curve vs - curve — when should you prefer PR over ROC?
Standardization vs normalization for scaling — what's the difference, and which models actually need scaled features vs which don't?
Beyond simple mean/median imputation, what are practical strategies for handling missing data and outliers?
What is multicollinearity, and how does VIF ( Inflation Factor) help detect it? Why does it hurt a linear 's interpretability?
How does ensemble stacking differ from and ?
What is the difference between a generative and a discriminative ? Give an example of each.
What is the , and why does it hurt distance-based models like ?