Mark Hall introduces a plugin that runs Weka on a cluster of machines. It uses the “map-reduce” framework, and operates with both Spark and Hadoop.
Share this step
Mark Hall from Pentaho introduces a plugin that runs Weka on a cluster of machines. It uses the “map-reduce” framework, and operates with both Spark and Hadoop. It comprises two Weka packages, distributedWekaBase, which provides general map-reduce tasks for machine learning that are not tied to any particular map-reduce implementation, and distributedWekaSpark, a wrapper for the base tasks that operates on the Spark platform. (There are also packages for Hadoop.) The aim is to support all Weka’s classification and regression algorithms without reimplementing them, generating output just like that produced by standard Weka. Clustering, however, had to be rewritten specifically for the distributed framework.