Repeat part (c) for model M1 using the threshold t = 0.1. Which threshold do you prefer, t = 0.5 or t = 0.1? Are the results consistent with what you expect from the ROC curve?

You are asked to evaluate the performance of two classification models, M1

and M2. The test set you have chosen contains 26 binary attributes, labeled

as A through Z.

Table 5.5 shows the posterior probabilities obtained by applying the models to

the test set. (Only the posterior probabilities for the positive class are shown).

As this is a two-class problem, P(?)=1 ? P(+) and P(?|A, . . . , Z)=1 ?

P(+|A, . . . , Z). Assume that we are mostly interested in detecting instances

from the positive class

When t = 0.1, the confusion matrix for M1 is shown below.





For t = 0.5, area = 0.6 × (1 ? 0.2) = 0.6 × 0.8=0.48.

For t = 0.1, area = 1 × (1 ? 0.8) = 1 × 0.2=0.2.

Since the area for t = 0.5 is larger than the area for t = 0.1, we prefer

t = 0.5.