Sandhya Indurkar

Math foundations

Feature Scaling: Why Dollars Crush Session Counts Without Z-Scores

Raw feature space versus z-scored space for distance

The idea

Machine learning rows are vectors. When one feature is revenue in dollars and another is session count, Euclidean distance mostly measures the dollar gap. The smaller feature barely votes. That breaks k-NN, clustering, and any model that compares rows by distance.

Feature scaling fixes the unit mismatch. Z-scores center each column at its mean and divide by standard deviation so a one-unit move means one standard deviation in that feature. Min-max scaling squeezes values into a fixed range like 0 to 1 when you need bounded inputs.

Scale before distance: otherwise the largest-magnitude feature decides who is nearest.

Example: L2 distance before and after z-scoring

Two features on different scales make distance follow the loud dimension. Z-scores put both on comparable footing so k-NN and clustering see both signals.

Revenue in dollars dwarfs session counts. Distance follows the loud feature.

Point A (Revenue, Sessions)

Point B

Z-score parameters

Raw space (orange axis stretch)

RevenueSessions

Z-scored space (balanced)

z1z2

L2 distance comparison

RawZ-score60.411.61

Raw L2

60.41

Z-scored L2

1.61

Raw L2 distance is 60.41 because Revenue gaps dominate. After z-scoring, L2 drops to 1.61 and both features share influence.

The math

Z-score

z = (x - mu) / sigma

Subtract the column mean mu, divide by standard deviation sigma. After z-scoring, each feature has mean 0 and standard deviation 1 on the training sample.

Min-max scaling

x_scaled = (x - min) / (max - min)

Maps the observed range to 0 through 1. Useful when you need bounded inputs or interpretable 0/1 anchors. Sensitive to outliers that stretch min or max.

Why distance breaks

d2(a, b) = sqrt(sum (ai - bi)^2)

L2 distance adds squared gaps across features. A $500 gap dominates a 5-session gap unless you scale first. Z-scores make each coordinate contribute in standard-deviation units.

A simple application

Before you ship a similarity search or k-NN classifier, list your features and their units. If magnitudes differ by orders of magnitude, z-score or min-max on the training set, then apply the same transform at scoring time.

Pair this with the norms-and-distance and k-NN posts: scaling is the step that makes those formulas fair across columns.