spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kyle Heath (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-2868) Support named accumulators in Python
Date Wed, 07 Jun 2017 20:29:18 GMT

    [ https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041557#comment-16041557
] 

Kyle Heath commented on SPARK-2868:
-----------------------------------

@[~holdenk]: I would love to better understand the scope of the work if you have time to sketch
it out for me.

> Support named accumulators in Python
> ------------------------------------
>
>                 Key: SPARK-2868
>                 URL: https://issues.apache.org/jira/browse/SPARK-2868
>             Project: Spark
>          Issue Type: New Feature
>          Components: PySpark
>            Reporter: Patrick Wendell
>
> SPARK-2380 added this for Java/Scala. To allow this in Python we'll need to make some
additional changes. One potential path is to have a 1:1 correspondence with Scala accumulators
(instead of a one-to-many). A challenge is exposing the stringified values of the accumulators
to the Scala code.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message