1. ROC Curve:
- The ROC curve is a graphical representation that shows the diagnostic ability of a binary classification model at various threshold settings.
-
It plots two metrics:
- True Positive Rate (TPR), also called Sensitivity or Recall, on the y-axis.
- False Positive Rate (FPR), on the x-axis.
- TPR (Sensitivity, Recall) = TPTP+FN\frac{TP}{TP + FN}
- FPR = FPFP+TN\frac{FP}{FP + TN}
Where:
- TP (True Positives): Correctly predicted positive cases.
- TN (True Negatives): Correctly predicted negative cases.
- FP (False Positives): Negative cases incorrectly predicted as positive.
- FN (False Negatives): Positive cases incorrectly predicted as negative.
The ROC curve shows how the model’s performance changes as the decision threshold (i.e., the probability cutoff) changes. For example, if the threshold is set very low, the model will classify many instances as positive, leading to a higher TPR but also a higher FPR.
2. AUC (Area Under the Curve):
- AUC stands for Area Under the Curve, which refers to the area under the ROC curve.
- AUC is a single scalar value that quantifies the overall performance of the classifier.
-
The value of AUC ranges from 0 to 1:
- AUC = 0.5: The model is no better than random guessing.
- AUC = 1: The model perfectly distinguishes between the positive and negative classes.
- AUC < 0.5: The model is worse than random guessing (this can happen if the classifier is flipped, i.e., it predicts positive when it should predict negative and vice versa).
AUC is useful for comparing different models. Higher AUC generally indicates a better model, as it suggests the model is more capable of distinguishing between the positive and negative classes.
Key Points:
- ROC is a curve that shows the tradeoff between sensitivity (recall) and the false positive rate at different thresholds.
- AUC is a summary statistic that gives you a single number to gauge the model’s performance across all thresholds.
Would you like to see an example of how the ROC curve is generated and how to compute AUC in Python?
.