spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-11904) pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing
Date Sat, 21 Nov 2015 22:58:10 GMT

     [ https://issues.apache.org/jira/browse/SPARK-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-11904:
------------------------------------

    Assignee:     (was: Apache Spark)

> pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing
> ---------------------------------------------------------------------
>
>                 Key: SPARK-11904
>                 URL: https://issues.apache.org/jira/browse/SPARK-11904
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 1.5.2
>            Reporter: David Tolpin
>
> When invFunc argument of reduceByKeyAndWindow is None, checkpointing should not be required.
Scala implementation of reduceByKeyAndWindow handles this correctly and does not require checkpointing.
Python version requires that checkpoint directory be specified, it is a bug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message