spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Commented] (SPARK-15574) Python meta-algorithms in Scala
Date Tue, 21 Jun 2016 00:42:57 GMT


Apache Spark commented on SPARK-15574:

User 'yinxusen' has created a pull request for this issue:

> Python meta-algorithms in Scala
> -------------------------------
>                 Key: SPARK-15574
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML, PySpark
>            Reporter: Joseph K. Bradley
> This is an experimental idea for implementing Python ML meta-algorithms (CrossValidator,
TrainValidationSplit, Pipeline, OneVsRest, etc.) in Scala.  This would require a Scala wrapper
for algorithms implemented in Python, somewhat analogous to Python UDFs.
> The benefit of this change would be that we could avoid currently awkward conversions
between Scala/Python meta-algorithms required for persistence.  It would let us have full
support for Python persistence and would generally simplify the implementation within MLlib.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message