spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangrui Meng <men...@gmail.com>
Subject Re: [MLlib] Contributing Algorithm for Outlier Detection
Date Tue, 21 Oct 2014 16:40:00 GMT
Hi Ashutosh,

The process you described is correct, with details documented in
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark
. There is no outlier detection algorithm in MLlib. Before you start
coding, please open an JIRA and let's discuss which algorithms are
appropriate to include, because there are many outlier detection
algorithms. I'm not sure which one is general enough and easy to
implement in parallel. For example, I'm not familiar with the
algorithm you mentioned, while the one I'm familiar with is based on
leverage scores: http://en.wikipedia.org/wiki/Leverage_(statistics)

Best,
Xiangrui

On Tue, Oct 21, 2014 at 2:23 AM, Ashutosh <ashutosh.trivedi@iiitb.org> wrote:
> Hi,
> I am new to Apache Spark (any open source project). I want to contribute to
> it. I found that MLlib has no algorithm for outlier detection yet.  By
> literature review I found the algorithm Attribute Value Frequency (AVF) is
> promising. Here is the link  DOI: 10.1109/ICTAI.2007.125
>
> By following the process I figured out that, I have to open a new feature
> request at JIRA (https://issues.apache.org/jira/browse/SPARK). Also, I have
> checked that no other issue is opened on "outlier detection".
>
> I want to know is it the right way to go? What project owners have in mind
> about outlier detection? Also is anybody working on parallel K nearest
> neighbour?
>
> Apart from opening up the feature request then pull request from git, How to
> provide the test cases?
>
> Suggestions and guidance are welcome.
>
> Thanks,
> Ashutosh
>
>
>
> --
> View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/MLlib-Contributing-Algorithm-for-Outlier-Detection-tp8880.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message