Return-Path: X-Original-To: apmail-spark-issues-archive@minotaur.apache.org Delivered-To: apmail-spark-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 42AF818C00 for ; Sat, 21 Nov 2015 22:58:11 +0000 (UTC) Received: (qmail 92747 invoked by uid 500); 21 Nov 2015 22:58:11 -0000 Delivered-To: apmail-spark-issues-archive@spark.apache.org Received: (qmail 92715 invoked by uid 500); 21 Nov 2015 22:58:11 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 92702 invoked by uid 99); 21 Nov 2015 22:58:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 21 Nov 2015 22:58:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id E5B852C1F6A for ; Sat, 21 Nov 2015 22:58:10 +0000 (UTC) Date: Sat, 21 Nov 2015 22:58:10 +0000 (UTC) From: "Apache Spark (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (SPARK-11904) pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SPARK-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11904: ------------------------------------ Assignee: (was: Apache Spark) > pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing > --------------------------------------------------------------------- > > Key: SPARK-11904 > URL: https://issues.apache.org/jira/browse/SPARK-11904 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 1.5.2 > Reporter: David Tolpin > > When invFunc argument of reduceByKeyAndWindow is None, checkpointing should not be required. Scala implementation of reduceByKeyAndWindow handles this correctly and does not require checkpointing. Python version requires that checkpoint directory be specified, it is a bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org