Data Mining with Weka

Syllabus

• Week 1

  A little bit of everything

  • What's data mining? What's Weka? What's the course about?

    Everybody talks about data mining and "big data" nowadays. This course introduces you to practical data mining. Weka is a powerful yet easy to use tool for machine learning and data mining that can also tackle large problems.

  • What's it like to do data mining?

    Each week we’ll focus on a “Big Question” relating to data mining. This is the question for this week.

  • Exploring the Explorer

    In this activity you will install Weka on your computer, start the Weka Explorer, and load, view, and edit datasets. (Note: You may need admin access to install Weka.)

  • Exploring datasets

    Classification (also called "supervised learning") is a common kind of data mining problem. Datasets contain "instances," which are described in terms of a fixed set of features, or "attributes".

  • Building a classifier

    Now you will learn how to use Weka's popular J48 classifier, which builds decision trees. J48 is a reimplementation of a classic classifier algorithm called C4.5.

  • Using a filter

    WEKA contains "filters" that help with cleaning and preparing your data. Some filters operate on attributes; others operate on instances.

  • Visualizing your data

    With Weka's visualization tool you can clean the data and remove anomalous instances (outliers). You can also visualize the errors that classifiers make.

• Week 2

  Evaluation

  • How do I evaluate a classifier’s performance?

    This week's Big Question!

  • Be a classifier!

    WEKA incorporates many different classification algorithms. One, called the “UserClassifier,” enables you to build your own decision tree for classification. How well can you do? It’s a challenge!

  • Training and testing

    Evaluating what has been learned is an essential part of data mining. You should never evaluate on the training set! – the results will be overly optimistic. If you have a single dataset, hold some data back for testing.

  • Repeated training and testing

    Ideally, training and test sets are sampled independently from a large population. Different samples give slightly different performance estimates. More reliable results are obtained by averaging over several experimental runs.

  • Baseline accuracy

    How do you know how well your machine learning method is doing? You should always compare it with the “baseline accuracy” obtained by simple methods. ZeroR is an extremely simple method that serves as a useful baseline.

  • Cross-validation

    Cross-validation, a standard evaluation technique, is a systematic way of running repeated percentage splits. In “stratified” cross-validation, training and test sets have the same class distribution as the full dataset.

  • Cross-validation results

    Cross-validation is better than randomly repeating percentage split evaluations. It gives a more reliable performance estimate – that is, one with lower variance. Ten-fold cross-validation is a standard evaluation method.

  • How are you getting on?

    We're well into the course now. Let's just take stock.

• Week 3

  Simple classifiers

  • How do simple classification methods work?

    This week's Big Question!

  • Simplicity first

    Always try simple methods before complex ones! (A good maxim for life in general, not just data mining.) Sometimes, simple algorithms perform really well. We learn about OneR, a simple method that is sometimes quite effective.

  • Overfitting

    “Overfitting” is a general problem that plagues all machine learning methods. It’s when a classifier fits the training data too tightly. The classifier works well on the training data but not on independent test data.

  • Using probabilities

    Why not use all attributes, equally weighted, instead of a single one as OneR does. Bayes' Theorem provides a sound probabilistic foundation for this. "Naive" Bayes assumes that attributes are equally important, and independent.

  • Decision trees

    Decision trees are another simple classification method, based on a top-down, recursive, divide-and-conquer strategy. J48 (aka C4.5) finds a good attribute to split on at each stage using a measure called "information gain."

  • Pruning decision trees

    Decision trees can easily overfit the training data, and pruning techniques are needed to guard against overfitting. There are various different methods. Unfortunately, this is where elegant algorithms get messy!

  • Nearest neighbor

    How about storing the training instances and giving new instances the same classification as their nearest neighbor? A similarity function is needed to select the closest instance. Using several neighbors can improve performance.

• Week 4

  More classifiers

  • What about real-life classification methods?

    This week's Big Question!

  • Classification boundaries

    Different classifiers are biased towards different kinds of decision, which you can explore by visualizing the classification boundaries. We look at classification boundaries for OneR, IBk, NaiveBayes, and J48.

  • Linear regression

    "Regression" problems are where the class is numeric, and "linear regression" is a standard mathematical technique for predicting numeric classes. In addition, there are non-linear methods that build trees of linear models.

  • Classification by regression

    Linear regression can be used for classification as well. For two-valued nominal classes, just convert them to 0 and 1. For more class labels, either "multi-response linear regression" or "pairwise linear regression" can be used.

  • Logistic regression

    Sometimes it’s best to predict class probabilities instead of predicting the classes themselves. Linear regression can be made to work with probabilities, resulting in logistic regression, a popular classification technique.

  • Support vector machines

    Support vector machines separate the classes using the "maximum margin hyperplane." This is defined by a few instances, called "support vectors," from each class. The boundary depends on a few points, which reduces overfitting.

  • Ensemble learning

    Many of us dislike committees, but nevertheless they often make good, unbiased, decisions. Several machine learning methods use "committees" of different classifier algorithms: Bagging, Random forests, Boosting, and Stacking.

• Week 5

  Putting it all together

  • What else is there to know?

    This week's Big Question!

  • The data mining process

    Producing classifiers is just a small part of the overall data mining process – perhaps the easiest part! Other parts involve formulating the question, gathering data, cleaning it, defining new features, and deploying the result.

  • Pitfalls and pratfalls

    Be skeptical, and particularly wary of overfitting. Missing values can signify various things; classifiers treat them differently. There’s no single "best learner"; all methods have biases. Data mining is an experimental science!

  • Data mining and ethics

    It’s far harder to anonymize data than you think! The purpose of data mining is to discriminate, but some kinds of discrimination are unethical, and illegal. Data mining discovers correlations, but these do not imply causation.

  • There's no magic in data mining

    There’s no magic in data mining! – in fact, perhaps Weka makes things too easy. You’ve learned lots, but don’t be smug: this course has missed out plenty. And you've learned a powerful technology: please use it wisely.

  • Farewell

    It's time to say goodbye.

Who will you learn with?