Glossary

A/B test

Randomly splitting users into groups shown different versions, to measure which version performs better in the real world.

ablation

An experiment that removes one component of a system at a time to measure how much that component actually contributes.

accuracy

The fraction of all predictions the model got right — misleading on imbalanced data (see precision/recall).

activation function

A non-linear function applied after a neuron's weighted sum (e.g. ReLU, sigmoid) — without it, a deep network would collapse into one linear function.

autoencoder

A network trained to compress its input through a bottleneck and reconstruct it — inputs that reconstruct poorly don't fit the learned "normal" pattern, which is useful for anomaly detection.

backpropagation

The algorithm that computes how much each weight contributed to the error, working backward from the output using the chain rule, so gradient descent knows how to adjust them.

bagging

Training many models independently on random samples of the data and averaging their predictions, to reduce variance.

baseline

A simple reference model or method that a more complex approach must beat to justify its added complexity.

batch

A small chunk of training examples processed together before the model's weights are updated once.

batch normalization

Normalizes each layer's inputs within a mini-batch during training, stabilizing and speeding up learning.

Bayes' theorem

A formula for updating the probability of something being true given new evidence: P(A|B) = P(B|A)P(A)/P(B).

benchmark

A standard dataset and evaluation setup used to compare different models/methods fairly.

Bernoulli distribution

The distribution of a single trial with two outcomes (success/failure) and a fixed probability of success.

BERT

A transformer pretrained by predicting masked (hidden) words using context from both directions — strong at understanding tasks like classification.

BFS

Explores a tree/graph level by level using a queue — finds the shortest path in unweighted graphs.

bias

Two common meanings: (1) a constant a neuron adds besides the weighted inputs, letting it shift its output; (2) in "bias-variance," the error from a model that is too simple to capture the real pattern.

Big-O

Notation describing how an algorithm's runtime or memory use grows as input size grows, ignoring constant factors.

binary search

A search algorithm on sorted data that repeatedly cuts the search space in half — O(log n).

boosting

Training models one after another, where each new model focuses on fixing the mistakes of the previous ones, to reduce bias.

Central Limit Theorem

States that the average of many independent samples tends toward a normal distribution, regardless of the original data's shape.

chain rule

A calculus rule for differentiating a composition of functions — the derivative of the outer function times the derivative of the inner one. This is the math behind backpropagation.

class imbalance

When one class (e.g. "fraud") is far rarer than another (e.g. "not fraud") in the training data, which breaks naive accuracy-based evaluation.

CNN

A neural network architecture that uses small shared filters sliding over the input — well-suited to images because it exploits spatial locality.

cold start

The problem of making good predictions for a new user/item with no history yet to learn from.

confounder

A hidden third variable that influences two others, making them look related when neither causes the other — e.g. summer driving both ice cream sales and drownings.

confusion matrix

A table comparing predicted vs actual classes, broken into true positives, false positives, true negatives, and false negatives.

cross-validation

Splitting data into multiple folds, training/testing on different fold combinations, and averaging the scores for a more reliable performance estimate.

curse of dimensionality

As feature count grows, data becomes exponentially sparser and distances less meaningful — so models (especially distance-based ones) need far more data.

data leakage

When information that shouldn't be available at prediction time (often from the test set) accidentally influences training, making performance look better than it really is.

DataFrame

A pandas table structure with labeled rows and columns, similar to a spreadsheet or a SQL table.

decision tree

A model that predicts by asking a series of yes/no questions about the features, branching down to a final answer — easy to visualize, prone to overfitting alone.

derivative

The instantaneous rate of change of a function — how much the output changes for a tiny nudge in the input (the slope).

DFS

Explores as far as possible down one branch before backtracking, typically using recursion or a stack.

dot product

A single number combining how large two vectors are and how aligned their directions are — large and positive when they point the same way.

dropout

A training trick where random neurons are temporarily switched off each pass, forcing the network not to over-rely on any single neuron.

early stopping

Stopping training once validation performance stops improving, to avoid overfitting to the training set.

eigenvalue

The factor by which a matrix stretches or shrinks its eigenvector.

eigenvector

A direction that doesn't rotate when a matrix transforms it — it only gets scaled by the matrix's eigenvalue.

epoch

One complete pass through the entire training dataset.

expectation

The probability-weighted average value of a random variable — its long-run "center."

exploding gradient

When gradients grow uncontrollably large as they propagate back through many layers, destabilizing training.

F1 score

The harmonic mean of precision and recall — a single number that punishes a model for being very good at one but very bad at the other.

feature

A measurable input variable the model uses to make a prediction — e.g. transaction amount, time of day, or user age.

forward pass

Running input data through a network layer by layer to produce a prediction.

GPT

A transformer pretrained by predicting the next word given only preceding context — strong at generation tasks like chat and text completion.

gradient boosting

Builds trees one at a time, each new tree correcting the errors of the ones before it — usually more accurate than random forest but easier to overfit.

gradient descent

An optimization algorithm that repeatedly nudges a model's weights in the direction that reduces the loss the fastest.

GROUP BY

A SQL clause that groups rows sharing the same value in a column, so aggregate functions (SUM, COUNT, AVG) can be applied per group.

hallucination

When an LLM confidently generates fluent but false content — it's trained to produce plausible text, not to verify facts.

hash map

A data structure that maps keys to values using a hash function, giving average O(1) lookup, insert, and delete.

hyperparameter

A setting chosen before training (like learning rate or tree depth) rather than learned from data.

IoU

A measure of overlap between two bounding boxes: the area they share divided by the total area they cover together.

JOIN

A SQL operation that combines rows from two or more tables based on a related column.

K-means

An unsupervised algorithm that groups data into K clusters by repeatedly assigning points to the nearest cluster center and recomputing centers.

kernel trick

A shortcut that lets algorithms like SVM operate as if data were mapped to a higher dimension, without ever computing that mapping explicitly.

KNN

Classifies a new point by looking at its K closest points in the training data and taking a majority vote of their labels.

L1 regularization

Penalizes the sum of absolute weight values; tends to push some weights to exactly zero, effectively selecting features.

L2 regularization

Penalizes the sum of squared weight values; shrinks all weights smoothly toward zero without eliminating them.

label

The correct answer attached to a training example — e.g. "spam" or "not spam" next to an email.

latency

The time delay between a request and its response — how long one prediction takes.

learning rate

How big a step gradient descent takes on each update — too high overshoots, too low is painfully slow.

lemmatization

Reducing a word to its actual dictionary base form using grammar/vocabulary (e.g. "better" → "good") — slower than stemming but linguistically correct.

linear regression

A model that predicts a continuous number as a weighted sum of input features plus a bias term.

logistic regression

A model for binary classification that outputs a probability using the sigmoid function on a weighted sum of features.

loss function

A number that measures how wrong the model's predictions are — training means finding weights that make this number as small as possible.

LSTM

A type of RNN with internal "gates" that better preserve long-range information and resist vanishing gradients.

model

The mathematical function (with learned parameters) that turns input features into a prediction — what you get after training.

NER

Automatically finding and labeling real-world entities in text — names, dates, amounts, locations.

neural network

A model made of layers of simple units (neurons) connected by weights, loosely inspired by the brain, capable of learning complex patterns.

neuron

A single unit in a neural network that computes a weighted sum of its inputs, adds a bias, and passes the result through an activation function.

normal distribution

A symmetric, bell-shaped probability distribution that many natural measurements follow.

numpy

A Python library for fast, vectorized numerical operations on arrays, backed by compiled C code.

object tracking

Linking detections of the same object across video frames over time, assigning it a consistent ID.

OCR

Optical Character Recognition — extracting machine-readable text from an image, like a scanned ID or document.

overfitting

A model that has memorized noise/quirks of the training data instead of the real pattern — great training score, poor performance on new data.

pandas

A Python library for working with tabular data using DataFrames — filtering, grouping, joining, and cleaning data.

PCA

A technique that finds the directions of maximum variance in data and re-expresses it using fewer dimensions along those directions.

Poisson distribution

Describes the count of rare, independent events happening in a fixed interval of time or space.

precision

Of everything the model flagged as positive, what fraction was actually positive — TP / (TP + FP).

random forest

Many decision trees trained on random subsets of data/features, whose predictions are averaged/voted — reduces overfitting versus a single tree.

rank

The number of truly independent directions (rows/columns) of information a matrix contains.

recall

Of everything that was actually positive, what fraction the model caught — TP / (TP + FN).

regularization

Any technique that discourages a model from becoming too complex/overfit, usually by penalizing large weights during training.

reinforcement learning

An agent learns by taking actions in an environment and getting rewards or penalties, gradually improving its behavior through trial and error — like training a dog with treats.

ReLU

An activation function that outputs the input if positive, otherwise zero — max(0, x). Cheap to compute and the default choice for hidden layers.

residual connection

A shortcut that lets a layer's input skip ahead and be added to its output, helping gradients flow through very deep networks.

RNN

A neural network that processes sequences step by step, carrying a hidden state forward — the basis for LSTM/GRU, now mostly replaced by Transformers.

ROC-AUC

A score summarizing how well a model ranks positives above negatives across all possible thresholds — 1.0 is perfect, 0.5 is random guessing.

self-attention

A mechanism where each token looks at every other token in a sequence and weighs how relevant each one is, producing context-aware representations.

shadow deployment

Running a new model on live traffic in parallel without acting on its outputs, to compare it safely against the current model before rollout.

sigmoid

An S-shaped function that squashes any real number into the range (0, 1), used to turn raw scores into probabilities.

sliding window

A technique that maintains a contiguous range of elements and moves its boundaries to find the best subarray/substring in one pass.

SMOTE

A technique that synthesizes new, realistic minority-class examples to fight class imbalance, instead of just duplicating existing ones.

softmax

Turns a vector of raw scores into a probability distribution over multiple classes that sums to 1.

stemming

A crude, rule-based way of chopping words down to a root form (e.g. "studies" → "studi") — fast but can produce non-words.

stochastic gradient descent

Gradient descent that updates weights using just one (or a small batch of) example at a time instead of the whole dataset — faster but noisier.

subword tokenization

Breaking rare or unknown words into smaller known pieces (e.g. "un" + "happi" + "ness"), so the model never hits a totally unseen word.

supervised learning

Training a model on examples that already have the correct answer attached (labeled data), so it learns to predict that answer for new examples.

SVM

A classifier that finds the boundary line/plane that separates classes with the widest possible margin.

TF-IDF

A way to score how important a word is to a document, based on how often it appears there versus how common it is across all documents.

throughput

How many requests/predictions a system can handle per unit of time.

tokenization

Splitting text into smaller units (words, subwords, or characters) that a model can process.

transformer

A neural network architecture built on self-attention that processes an entire sequence at once — the basis of BERT, GPT, and most modern NLP.

two-pointer

A technique using two index variables that move through an array/string (often from both ends) to solve a problem in one pass.

underfitting

A model too simple to capture the real pattern in the data — performs poorly on both training and new data.

unsupervised learning

Finding patterns or structure in data that has no labels — e.g. grouping similar customers together without being told the groups in advance.

vanishing gradient

When gradients shrink toward zero as they propagate back through many layers, so early layers barely learn.

variance

How spread out a set of values is around their average. In "bias-variance," it means how much a model's predictions swing if trained on slightly different data (overly sensitive = high variance).

weak supervision

Using noisy, indirect, or rule-based signals to generate approximate labels when real labeled data isn't available.

weight

A learned number that controls how much a given input contributes to the model's output — training is the process of finding good weight values.

word embedding

A dense vector representation of a word such that semantically similar words end up close together in vector space.

zero-shot classification

Using a pretrained model to classify into categories it was never specifically trained on, by framing the task as a prompt.