Error Analysis and ROC

Given Maja’s concerns, what is the *most appropriate* measure to use and value based upon the data collected?

Incorrect. Accuracy is not the appropriate measure when the data is unequally distributed because the average will not be close to the true value.

Correct! The F1 score is the best measure when the data is unequally distributed and it is necessary to measure the equilibrium between precision and recall. The F1 score is calculated as: $$\displaystyle \text{F1} = \frac{2 \times P \times R}{(P + R)} = \frac{2 \times 0.8731 \times 0.9208}{0.8731 + 0.9208} \approx 0.90$$.

Incorrect. The false positive rate is not the appropriate measure because the unequal distribution will skew the false positive rate.

F1 score and 0.90

Accuracy and 0.86

False positive rate and 0.27