spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Egor Pahomov <pahomov.e...@gmail.com>
Subject Re: MLlib enable extension of the LabeledPoint class
Date Thu, 25 Sep 2014 14:27:29 GMT
I agree with Yu, that you should tell more about your intentions, but
possible dirty workaround is create wrapper class for LabeledPoint with all
additional information you need and unwrap values before train, and wrap
them again after. (look at zipWithIndex - it helps match back additional
information after unwrapping)

But I would rather patch my spark with method signature chagnes you
suggested.

2014-09-25 18:22 GMT+04:00 Egor Pahomov <pahomov.egor@gmail.com>:

> @Yu Ishikawa,
>
> *I think the right place for such discussion -
>  https://issues.apache.org/jira/browse/SPARK-3573
> <https://issues.apache.org/jira/browse/SPARK-3573>*
>
>
> 2014-09-25 18:02 GMT+04:00 Yu Ishikawa <yuu.ishikawa+spark@gmail.com>:
>
>> Hi Niklas Wilcke,
>>
>> As you said, it is difficult to extend LabeledPoint class in
>> mllib.regression.
>> Do you want to extend LabeledPoint class in order to use any other type
>> exclude Double type?
>> If you have your code on Github, could you show us it? I want to know what
>> you want to do.
>>
>> > Community
>> By the way, I think LabeledPoint class is very useful exclude
>> mllib.regression package.
>> Especially, some estimation algorithms should use a type for the labels
>> exclude Double type,
>> such as String type. The common generics labeled-point class would be
>> useful
>> in MLlib.
>> I'd like to get your thoughts on it.
>>
>> For example,
>> ```
>> abstract class LabeledPoint[T](label: T, features: Vector)
>> ```
>>
>> thanks
>>
>>
>>
>>
>>
>>
>> -----
>> -- Yu Ishikawa
>> --
>> View this message in context:
>> http://apache-spark-developers-list.1001551.n3.nabble.com/MLlib-enable-extension-of-the-LabeledPoint-class-tp8546p8549.html
>> Sent from the Apache Spark Developers List mailing list archive at
>> Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
>
> --
>
>
>
> *Sincerely yoursEgor PakhomovScala Developer, Yandex*
>



-- 



*Sincerely yoursEgor PakhomovScala Developer, Yandex*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message