Math, Applied
Base Rates and Updating Beliefs: Rare Events, Loud Alerts
The idea
A fraud model flags an order. A resume screen passes a candidate. A support ticket gets routed as high risk. Each signal feels strong. But if the underlying rate is rare, most flagged cases can still be false alarms.
The base rate is how common the thing is before you see any signal. Only 2% of orders might be fraud. A 90% accurate alert still leaves plenty of clean orders in the flagged pile when fraud is that uncommon.
Base rates answer: How common is this outcome in the full population, before this alert fired?
Example: update a belief after a positive signal
A strong alert is not the same as a near-certain hit when the underlying rate is rare. Adjust base rate and test accuracy to see the updated chance after a positive signal.
Base rate (before signal)
2.0%
Share of orders that are actually fraud
After positive signal
18.7%
Model flags an order
After a positive signal, the updated chance is 18.7%, up from a 2.0% base rate. Better, but not certainty.
The math
Bayes: update after a positive signal
P(A) is the base rate. P(signal | A) is true positive rate. P(signal) mixes real hits and false alarms. The explorer computes the posterior when a signal fires.
Why false alarms dominate when A is rare
With 2% fraud, 90% true positives, and 8% false positives on clean orders, most alerts still come from the huge pool of legitimate orders. The updated chance after an alert is far below 90%.
Better test accuracy helps, but it cannot erase a very low base rate by itself. You also need context: seasonality, segment, and what action you take on a flag. Updating beliefs is not skepticism. It is matching confidence to how common the outcome really is.
A simple application: fraud alerts
Risk and trust teams calibrate review queues. Recruiting teams pair screens with structured follow-ups instead of treating a pass as a hire. Support leaders set escalation rules that account for how often tickets truly need tier-two help.
Fraud alerts: base rate before you react
Move true fraud rate and alert precision. Most alerts can be false even when the model looks useful.
20 alerts/1k txn → ~7 true, ~13 false
Alerts per 1k txn
Rates (%)
Base fraud: 0.8% · Precision: 35.0%
Base fraud rate
0.8%
Alert precision
35%
False alerts/1k
~13
Optimize (move here)
- • State base rate out loud in queue reviews
- • Calibrate staffing to precision × volume
Hold (do not over-react)
- • Investigating every alert as if fraud were common
Escalate if
- • False alert cost exceeds fraud loss prevented
Only ~0.8% of txn are fraud. State base rate before staffing review queue.
The habit is stating the base rate out loud before you react to a dashboard alert. Then ask what a positive signal actually buys you. That keeps scarce reviewer time on cases where the math supports action.