spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "holdenk (JIRA)" <>
Subject [jira] [Commented] (SPARK-2868) Support named accumulators in Python
Date Fri, 07 Oct 2016 23:24:20 GMT


holdenk commented on SPARK-2868:

Is this something we are still interested in pursuing (cc [~rxin] who did the Scala accumulator
API update). I'd be happy to take on this issue, once we've decide what to do around data
property accumulators, since I've already been working with accumulators a bunch.

> Support named accumulators in Python
> ------------------------------------
>                 Key: SPARK-2868
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: PySpark
>            Reporter: Patrick Wendell
> SPARK-2380 added this for Java/Scala. To allow this in Python we'll need to make some
additional changes. One potential path is to have a 1:1 correspondence with Scala accumulators
(instead of a one-to-many). A challenge is exposing the stringified values of the accumulators
to the Scala code.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message