Skip main navigation

Cost-sensitive classification conclusions

Ian Witten discusses the effect of cost-sensitive evaluation and cost-sensitive classification on the credit-g dataset, using Naive Bayes and J48.

Even if you don’t do the exercise, you should look at the numbers and note that they support the following general conclusions for the credit-g dataset with a particular cost matrix.

  • Naive Bayes is generally better than J48 on this problem.

  • Using cost-sensitive classification or cost-sensitive learning always improves the result.

  • For Naive Bayes, cost-sensitive learning gives about the same result as cost-sensitive classification.

  • For J48, cost-sensitive learning gives a better result than cost-sensitive classification.

  • Bagging improves the result of cost-sensitive classification using J48.

(Note incidentally that the Experimenter differs from the Explorer in that it calculates the four elements of the confusion matrix, Num_false_positives etc, separately for each cross-validation fold and averages them, whereas the Explorer sums them over the entire dataset. Thus, since we are using 10-fold cross-validation in this activity, the numbers, and the costs, produced by the Experimenter are 10% of those produced by the Explorer. The Experimenter does this because a separate confusion matrix is needed for each fold in order to compute the standard deviation of each element and to do significance tests between corresponding elements produced by different classifiers.)

This article is from the free online

More Data Mining with Weka

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education