A fraud model says 0.95 on a $10,000 transaction. Can you read that as "95% chance of fraud"? Usually no. I trained a fraud model on real data to check. Here is the whole idea in five pictures.
1. The model says 0.95. Reality says 0.51.
real numbers: ULB credit card dataset (284,807 transactions, 492 frauds). gradient boosting trained on 1:5 undersampled data. on a 40% holdout, 330 transactions scored ≥ 0.95 and 167 of them (50.6%) were actually fraud.
2. Ranking is not measuring
automated decisions need the right side to be true.
3. The reliability curve shows the lie
green dashed line = trustworthy. blue curve = this model on the holdout. mid scores are even worse: bins from 0.5 to 0.9 are only 1-2% fraud. one number for the whole gap: ECE.
4. Fix: calibrate on data the model never saw
calibrating on training data just memorizes the miscalibration.
5. Now the number can make the decision
$5,100 still clears the line here, so block. a 0.51 on a $500 transaction would not.
The whole pipeline
not score → block.