ROC Curve & AUC calculator

Plot Receiver Operating Characteristic (ROC) curves online. Calculate Area Under the Curve (AUC), sensitivity, and specificity thresholds.

What it does

Plots sensitivity against 1−specificity across every possible cutoff. The area underneath is the probability that a randomly chosen case scores higher than a randomly chosen non-case.

When to use it

Judging how well a continuous marker or model score separates two groups, independently of any particular threshold.

Cautions

• AUC measures ranking only. A model can rank perfectly and still produce badly wrong probabilities.
• It is insensitive to class imbalance in a way that flatters rare-outcome models; precision-recall is often more informative there.
• AUC computed on the data used to fit the model is optimistic. Validate on data the model has not seen.

Alternatives

• A confusion matrix at a chosen cutoff — you are going to deploy one threshold — AUC averages over all of them and cannot tell you what a decision costs
• Calibration measures — you need the predicted probabilities to be accurate, not just correctly ordered

How to read the output

AUC

The probability that a randomly chosen case with the event has a higher score than a randomly chosen case without it. 0.5 is a coin toss; 1.0 is perfect separation. It is a ranking measure and depends on no threshold at all. AUC is not accuracy and cannot be read as 'right 80% of the time'. It is also insensitive to how common the outcome is, which is a strength for comparing scores and a trap for judging usefulness — a score with an AUC of 0.85 on a 1-in-500 outcome can still flag ten false positives for every true one.

The curve itself

Every point is one threshold. The top-left corner is perfect; the diagonal is worthless. The SHAPE matters as much as the area — a curve that rises steeply at the left reaches high sensitivity cheaply. Two scores with identical AUCs can have differently-shaped curves, and one of them may be much better in the region you would actually operate in. Compare the curves where you would set the cut-off, not just the summary number.

95% confidence interval (DeLong)

The interval around the AUC, from the same DeLong variance the two-curve comparison uses. It is what separates an AUC from 40 cases from one from 4,000, which the point estimate alone does not. It is computed on the AUC scale, so for a very strong classifier the upper bound can run past 1; it is clipped and labelled when that happens, which is a sign the normal approximation is straining rather than that performance is perfect. At AUC exactly 1 — perfect separation — the DeLong variance is zero and the interval collapses to [1, 1]. That is an artefact of a small sample in which no case overlaps any control, not evidence of certainty, and it is the first thing to distrust rather than the last. An interval overlapping another predictor's is not a test of the difference — use DeLong's test for that, since the two curves are correlated.

DeLong's test (two predictors)

Compares two AUCs measured on the SAME cases, accounting for the fact that they are correlated because they rank the same patients. Select exactly two predictors and it appears automatically. It tests whether the AUCs differ, not whether the difference is worth having. It is also only valid for scores computed on the same rows — comparing an AUC here with one from a published paper is not what this test does.

The threshold table and the confusion matrix

The operating points: at each cut-off, the counts and the resulting sensitivity, specificity, PPV, NPV, accuracy and Youden's J. This is where the curve turns into a decision. PPV and NPV depend on how common the outcome is in YOUR data. Carrying them to a population with a different prevalence is invalid, while sensitivity and specificity carry across.

Youden's J and the 'optimal' threshold

J = sensitivity + specificity − 1, and the marked cut-off is the one that maximises it. It is the standard summary choice. 'Optimal' here means optimal under the assumption that a false positive and a false negative cost the SAME. They almost never do. Choose the threshold from the consequences — a missed diagnosis against an unnecessary test — and treat Youden's as a default, not an answer. Choosing it on the same data also biases it optimistically.

Excluded rows

Rows missing the outcome or any selected predictor are dropped, and the count is reported. All predictors are evaluated on the same surviving rows, which is what makes DeLong's comparison legitimate. Adding a predictor with scattered missingness shrinks the sample for every curve on the plot, not just its own. If the AUCs move when you add a predictor to the selection, check the excluded count before believing it.

How this calculator is validated · Which statistical test should I use?