spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <>
Subject [jira] [Commented] (SPARK-21027) Parallel One vs. Rest Classifier
Date Fri, 09 Jun 2017 23:43:19 GMT


Joseph K. Bradley commented on SPARK-21027:

One other comment: The Scala implementation can be faster for some tasks but can be slower
for others (b/c of conflicting Spark jobs).  It'd be nice to permit the user to tune the parallelism,
with some reasonable default such as 4 parallel training jobs.

> Parallel One vs. Rest Classifier
> --------------------------------
>                 Key: SPARK-21027
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: PySpark
>    Affects Versions: 2.2.0, 2.2.1
>            Reporter: Ajay Saini
> Currently, the Scala implementation of OneVsRest allows the user to run a parallel implementation
in which each class is evaluated in a different thread. This implementation allows up to a
2X speedup as determined by experiments but is not currently not tunable. Furthermore, the
python implementation of OneVsRest does not parallelize at all. It would be useful to add
a parallel, tunable implementation of OneVsRest to the python library in order to speed up
the algorithm.
>  A ticket for the Scala implementation of this classifier is here:

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message