Confusion Matrix Calculator

Statistical measures based on the confusion matrix

The confusion matrix is the popular representation of the performance of classification models and includes the correctly and incorrectly classified values compared to the actual outcomes in the test data. The four variables are:

• True positive (TP) – which is the outcome where the model correctly predicts positive class (condition is correctly detected when present);
• True negative (TN) – which is the outcome where the model correctly predicts negative class (condition is not detected when absent);
• False positive (FP) – which is the outcome where the model incorrectly predicts positive class (condition is detected despite being absent);
• False negative (FN) – which is the outcome where the model incorrectly predicts negative class (condition is not detected despite being present).

One of the most commonly determined statistical measures is Sensitivity (also known as recall, hit rate or true positive rate TPR). Sensitivity measures the proportion of actual positives that are correctly identified as positives.

Sensitivity = TP / (TP + FN)

Specificity, also known as selectivity or true negative rate (TNR), measures the proportion of actual negatives that are correctly identified as negatives.

Specificity = TN / (FP + TN)

The Positive Predictive Value (PPV), also known as Precision and the Negative Predictive Value (NPV) are the proportion of positive and negative results that are true positive, respectively true negative. They are also called positive respectively negative predictive agreements and are measures of the performance of a diagnostic test.

Positive Predictive Value (Precision) = TP / (TP + FP)

Negative Predictive Value = TN / (TN + FN)

The False Positive Rate (FPR) or fall-out is the ratio between the number of negative events incorrectly categorized as positive (false positives) and the total number of actual negative events (regardless of classification).

False Positive Rate = FP / (FP + TN)

The False Discovery Rate (FDR) is a statistical approach used in multiple hypothesis testing to correct for multiple comparisons.

False Discovery Rate = FP / (FP + TP)

The False Negative Rate (FNR) measures the proportion of the individuals where a condition is present for which the test result is negative.

False Negative Rate = FN / (FN + TP)

Accuracy (ACC) is a measure of statistical bias

Accuracy = (TP + TN) / (TP + TN + FP + FN)

The F1 Score is a measure of a test’s accuracy, defined as the harmonic mean of precision and recall.

F1 Score = 2TP / (2TP + FP + FN)

Matthews Correlation Coefficient (MCC) describes how changing the value of one variable will affect the value of another and returns a value between -1 and 1:

• +1describes a perfect prediction;
• 0unable to return any valid information (no better than random prediction);
• -1describes complete inconsistency between prediction and observation.

Matthews Correlation Coefficient = (TP x TN – FP x FN) / (sqrt((TP+FP) x (TP+FN) x (TN+FP) x (TN+FN)))

References

Matthews, B. W. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Structure. 1975; 405 (2): 442–451.

Powers, David M W. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation (PDF). Journal of Machine Learning Technologies. 2011; 2 (1): 37–63.

Jakobsdottir J, Weeks DE. Estimating Prevalence, False-Positive Rate, and False-Negative Rate with Use of Repeated Testing When True Responses Are Unknown. Am J Hum Genet. 2007; 81(5): 1111–1113.

Lalkhen AG, McCluskye A. Clinical tests: sensitivity and specificity. Contin Educ Anaesth Crit Care Pain. 2008; 8(6): 221-223.