I’m sure that we all have heard of accuracy when comes to evaluating our Machine Learning model, especially for classifications.
In case you didn’t, it is simple. Let’s say we have a total of 100 coins, 80 of them are heads & 20 of them are tails.
If run our 100 coins through a machine-learning model, and see how many of them are predicted as heads and tails respectively, we can then evaluate the accuracy of our model.
Let’s say the machine learning model predicted: 78 heads & 22 tails, we know that 2 coins are predicted wrongly as 2 of them are predicted as tails but they should be heads. In this case, out of 100 coins, 98 of them are predicted correctly, we can then say the accuracy of this model is: 98/100 = 0.98 (98%).
In short, Accuracy = Correct Predictions / All Predictions
From the accuracy that we have talked about, the correct predictions can be broken down into 2 categories:
- True Negatives
- True Positives
Same for the incorrect predictions, they can be broken down into 2 categories:
- False Negatives
- False Positives
For example, there is a total of 10 people who went for a tumor diagnosis:
Let’s try to calculate accuracy again.
As we can see, there are a total of 7 correct predictions (6 predicted negatives & 1 predicted positive) that match the actual value. In this case, the Accuracy = (6 + 1) / 10 = 70%.
What about Precision & Recall?
Let’s start with precision. For precision, we look at only the positive predictions. In this case, there are a total of 3 of them.
Precision describes that out of all positive predictions, how many of them are really positive? As we can see, only 1 of them is the actual positive value.
In this case, our precision comes down to 1 / 3 = 0.333 (33.3%).
For recall, we look at only the real positive values. As we can see, there are a total of 2 of them.
Recall describes that out of all real positive values, how many of them are predicted as positive?
In this case, our recall comes down to 1 / 2 = 0.5 (50%).
Hope that wraps up for Precision & Recall.
As we can see, good accuracy does not mean good precision & recall.
This provides us the power to truly evaluate how our model is doing, especially when the sample size is small.