spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-21027) Parallel One vs. Rest Classifier
Date Fri, 09 Jun 2017 23:43:19 GMT

    [ https://issues.apache.org/jira/browse/SPARK-21027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045228#comment-16045228
] 

Joseph K. Bradley commented on SPARK-21027:
-------------------------------------------

One other comment: The Scala implementation can be faster for some tasks but can be slower
for others (b/c of conflicting Spark jobs).  It'd be nice to permit the user to tune the parallelism,
with some reasonable default such as 4 parallel training jobs.

> Parallel One vs. Rest Classifier
> --------------------------------
>
>                 Key: SPARK-21027
>                 URL: https://issues.apache.org/jira/browse/SPARK-21027
>             Project: Spark
>          Issue Type: New Feature
>          Components: PySpark
>    Affects Versions: 2.2.0, 2.2.1
>            Reporter: Ajay Saini
>
> Currently, the Scala implementation of OneVsRest allows the user to run a parallel implementation
in which each class is evaluated in a different thread. This implementation allows up to a
2X speedup as determined by experiments but is not currently not tunable. Furthermore, the
python implementation of OneVsRest does not parallelize at all. It would be useful to add
a parallel, tunable implementation of OneVsRest to the python library in order to speed up
the algorithm.
>  A ticket for the Scala implementation of this classifier is here: https://issues.apache.org/jira/browse/SPARK-21028



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message