Math, Applied
The New Users Look Great: Cohort Analysis in Real Decisions
The idea
Headline retention is a blend. It mixes users who signed up last week with users who signed up last quarter. When the blend shifts toward newer signups, the overall rate can move even if product quality for each signup month is unchanged.
Remember it in one line: compare signup cohorts at the same age before you react to the headline average.
Cohort analysis groups users by when they started, then tracks each group on the same clock. Did January signups retain better than December signups at day 30? That is a product question. Did overall retention fall because January brought twice as many signups? That is a mix question. A retention heatmap makes the pattern visible: each row is a signup month, each column is weeks since signup, and the color is how many users are still active.
Cohort analysis answers: Is the metric moving because behavior changed, or because who we are measuring changed?
Example: headline retention vs signup cohorts
Each row is day-30 retention for users who signed up in that month. The headline number is a weighted blend of those cohorts.
Signup-month quality is flat; the active base got younger.
Headline retention
42.7%
Weighted blend of cohort rows below
| Cohort | D30 retention | Mix share |
|---|---|---|
| Oct signup | 46.0% | 32% |
| Nov signup | 43.0% | 40% |
| Dec signup | 40.0% | 13% |
| Jan signup | 37.0% | 15% |
Cohort retention heatmap
Self-serve SaaS: % of signup cohort still active each week
| Cohort | Week 0 | Week 1 | Week 2 | Week 3 | Week 4 |
|---|---|---|---|---|---|
| Oct signup | 100% | 73% | 60% | 52% | 46% |
| Nov signup | 100% | 71% | 58% | 49% | 43% |
| Dec signup | 100% | 69% | 55% | 46% | 40% |
| Jan signup | 100% | 67% | 52% | 43% | 37% |
Read across a row for one cohort's decay curve. Compare rows at the same week column: Jan is not worse at week 4 because of mix alone, each cohort has its own row.
Headline retention is 42.7% with 28% of users from recent signups. Cohort D30 rates are unchanged. Check whether the mix shifted before you blame the product.
The math
Weighted blend
Overall retention is a weighted average of cohort rates. Weights are how many users from each signup month sit in the measured population today. Change the weights and the headline moves, even when every cohort row stays fixed.
Apples to apples
Jan signups at day 30 vs Dec signups at day 30 is a fair product comparison. Jan signups at day 7 vs Oct signups at day 90 is not. Lock the age window before you rank months.
Separate mix from quality
Hold cohort rows steady and only shift weights to see how much of a headline move is composition. Hold weights steady and move cohort rows to see real behavior change. Teams need both views in every retention review.
A simple application: the weekly retention review
Growth reports headline retention at 44%, down two points. The cohort table shows Oct through Jan signup months each two points lower at day 30 than the prior month. That is a product regression. If cohort rows are flat but January doubled signups, the headline drop is a mix story, not necessarily a broken feature.
Weekly retention review: headline vs cohort rows
Shift January signup mix. Headline retention can fall while each cohort row stays flat.
Headline 45% retention — older cohorts still ~48%
Retention levels
Older cohorts: 48% · New cohort: 38% · Headline: 45%
Mix vs quality
New cohort %: 28 · Ret gap (pp): 10
Headline D30
45%
Older cohorts
48%
New cohort share
28%
Optimize (move here)
- • Publish cohort table beside headline retention
- • Separate mix from quality moves
Hold (do not over-react)
- • Roadmap pivots from headline alone after signup surge
Escalate if
- • Each cohort row falls quarter over quarter
Mix shift: more users from a lower-retention cohort. Check cohort table before calling a regression.
The habit: publish a cohort table beside every headline retention metric. Flag mix shifts after launches and campaigns. Pair with segment tables when experiments change who signs up. Weighted average posts cover rollups. This post covers who belongs in each row.