Return-Path: X-Original-To: apmail-spark-commits-archive@minotaur.apache.org Delivered-To: apmail-spark-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C15D11D12 for ; Sat, 9 Aug 2014 03:58:47 +0000 (UTC) Received: (qmail 61468 invoked by uid 500); 9 Aug 2014 03:58:47 -0000 Delivered-To: apmail-spark-commits-archive@spark.apache.org Received: (qmail 61431 invoked by uid 500); 9 Aug 2014 03:58:47 -0000 Mailing-List: contact commits-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list commits@spark.apache.org Received: (qmail 61422 invoked by uid 99); 9 Aug 2014 03:58:47 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 09 Aug 2014 03:58:47 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 14E5B95219E; Sat, 9 Aug 2014 03:58:47 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rxin@apache.org To: commits@spark.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: git commit: [SPARK-2911] apply parent[T](j) to clarify UnionRDD code Date: Sat, 9 Aug 2014 03:58:47 +0000 (UTC) Repository: spark Updated Branches: refs/heads/master 1c84dba98 -> 43af28170 [SPARK-2911] apply parent[T](j) to clarify UnionRDD code References to dependencies(j) for actually obtaining RDD parents are less common than I originally estimated. It does clarify UnionRDD (also will clarify some of my other PRs) Use of firstParent[T] is ubiquitous, but not as sure that benefits from being replaced with parent(0)[T]. Author: Erik Erlandson Closes #1858 from erikerlandson/spark-2911-pr2 and squashes the following commits: 7ffea74 [Erik Erlandson] [SPARK-2911] apply parent[T](j) to clarify UnionRDD code Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/43af2817 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/43af2817 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/43af2817 Branch: refs/heads/master Commit: 43af2817007eaa2cce2567bd83f5cde1ee28d1f7 Parents: 1c84dba Author: Erik Erlandson Authored: Fri Aug 8 20:58:44 2014 -0700 Committer: Reynold Xin Committed: Fri Aug 8 20:58:44 2014 -0700 ---------------------------------------------------------------------- core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/43af2817/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala index 197167e..0c97eb0 100644 --- a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala +++ b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala @@ -83,8 +83,7 @@ class UnionRDD[T: ClassTag]( override def compute(s: Partition, context: TaskContext): Iterator[T] = { val part = s.asInstanceOf[UnionPartition[T]] - val parentRdd = dependencies(part.parentRddIndex).rdd.asInstanceOf[RDD[T]] - parentRdd.iterator(part.parentPartition, context) + parent[T](part.parentRddIndex).iterator(part.parentPartition, context) } override def getPreferredLocations(s: Partition): Seq[String] = --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org For additional commands, e-mail: commits-help@spark.apache.org