mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: Taste Vs Weka
Date Wed, 27 Aug 2008 14:33:18 GMT

On Aug 27, 2008, at 8:33 AM, Richard Tomsett wrote:

> There's quite a good description of WEKA and its capabilities on the  
> course page for a module I took this year: http://www.inf.ed.ac.uk/teaching/courses/dme/html/software2.html
>
> It's more a general suite of data-mining tools rather than a tool to  
> address a specific task like Taste (plus it's obviously not  
> implemented for parallel processing which could be problematic for  
> scaling up). From the link above:
>
>   * *Advantages*: The obvious advantage of a package like Weka is that
>     *a whole range of data preparation, feature selection and data
>     mining algorithms are integrated*. This means that only one data
>     format is needed, and trying out and comparing different
>     approaches becomes really easy. The package also comes with *a
>     GUI*, which should make it easier to use.

Yeah, it would be good for Mahout to adopt an approach for either  
translating from ARFF to our format, or just use ARFF or whatever else  
Weka does, but I don't want it to preclude us from innovating where we  
need to innovate.


>
>
>   * *Disadvantages*: Probably the most important disadvantage of data
>     mining suites like this is that *they do not implement the newest
>     techniques*. For example the MLP implemented has a very basic
>     training algorithm (backprop with momentum), and the SVM only uses
>     polynomial kernels, and does not support numeric estimation. ...
>     *A third possible problem is scaling*. For difficult tasks on
>     large datasets, the running time can become quite long, and java
>     sometimes gives an OutOfMemory error. This problem can be reduced
>     by using the '-mx/x/' option when calling java, where /x/ is
>     memory size (eg '50m'). For large datasets it will always be
>     necessary to reduce the size to be able to work within reasonable
>     time limits. A fourth problem is that *the GUI does not implement
>     all the possible options*. Things that could be very useful, like
>     scoring of a test set, are not provided in the GUI, but can be
>     called from the command line interface. So sometimes it will be
>     necessary to switch between GUI and command line. Finally, *the
>     data preparation and visualisation techniques offered might not be
>     enough*. Most of them are very useful, but I think in most data
>     mining tasks you will need more to get to know the data well and
>     to get it in the right format.
>

 From a Mahout view, we are very much aiming at addressing the scaling  
issue.  As for the GUI, I think that will always be a "contrib" for  
Mahout, if one ever exists.  My personal goal for Mahout is to keep it  
lean and easily usable in a wide variety of applications.  Just as  
Lucene has made search a commodity in many ways, I think Mahout could  
enable ML to be a commodity in 5 years.

Also, a glaring difference between the two is Weka is GPL.  I'll leave  
it to you to read all the discussions on ASL vs. GPL and do not want  
to start that discussion here, as there is no point.

Last, I imagine we will all coexist nicely.  Weka will be useful for  
many tasks, and Mahout will be useful for many tasks and there will  
certainly be overlap.


Mime
View raw message