Consider the following approach for testing whether a classifier A beats an- other classifier B. Let N be the size of a given data set, pA be the accuracy of classifier A, pB be the accuracy of classifier B, and p = (pA + pB)/2 be the average accuracy for both classifiers. To test whether classifier A is significantly better than B, the following Z-statistic is used:

Classifier A is assumed to be better than classifier B if Z > 1.96.


Table 4.3 compares the accuracies of three different classifiers, decision tree


classifiers, na ??ve Bayes classifiers, and support vector machines, on various


data sets. (The latter two classifiers are described in Chapter 5.)

A summary of the relative performance of the classifiers is given below: