Math foundations
Orthogonality: When Two Features Carry Separate Signal
The idea
In vector space, orthogonal directions share no overlap. For standardized features, zero correlation means the inputs point at a 90° angle. Regression can assign credit cleanly because each column explains a different slice of variance. High correlation means nearly parallel vectors and shared credit.
Orthogonality answers: Are these inputs independent enough that each coefficient reads as its own lever?
Example: correlation as angle between feature vectors
Orthogonal features (90° apart) carry independent signal. Parallel features share the same story.
Randomized arms give near-orthogonal inputs. Coefficients read cleanly.
Angle from orthogonal
87°
Read
Near orthogonal
Correlation 5% → angle ~87° from orthogonal. Variant A flag and Variant B flag carry separate signal; coefficients are easier to read.
The math
Orthogonal vectors
Dot product zero when vectors are perpendicular. No shared directional component.
Uncorrelated columns
On centered, scaled features, zero correlation matches orthogonality in feature space.
Correlation as angle
r = 1 means parallel (same story). r = 0 means orthogonal (separate signal).
A simple application
Randomized A/B arms give near-orthogonal treatment flags. Correlated KPIs like revenue and order count do not. Check correlation heatmaps before you interpret multi-driver regression. When r is high, see multicollinearity and ridge posts.