spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject git commit: [SPARK-2911] apply parent[T](j) to clarify UnionRDD code
Date Sat, 09 Aug 2014 03:58:47 GMT
Repository: spark
Updated Branches:
  refs/heads/master 1c84dba98 -> 43af28170


[SPARK-2911] apply parent[T](j) to clarify UnionRDD code

References to dependencies(j) for actually obtaining RDD parents are less common than I originally
estimated.   It does clarify UnionRDD (also will clarify some of my other PRs)

Use of firstParent[T] is ubiquitous, but not as sure that benefits from being replaced with
parent(0)[T].

Author: Erik Erlandson <eerlands@redhat.com>

Closes #1858 from erikerlandson/spark-2911-pr2 and squashes the following commits:

7ffea74 [Erik Erlandson] [SPARK-2911] apply parent[T](j) to clarify UnionRDD code


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/43af2817
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/43af2817
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/43af2817

Branch: refs/heads/master
Commit: 43af2817007eaa2cce2567bd83f5cde1ee28d1f7
Parents: 1c84dba
Author: Erik Erlandson <eerlands@redhat.com>
Authored: Fri Aug 8 20:58:44 2014 -0700
Committer: Reynold Xin <rxin@apache.org>
Committed: Fri Aug 8 20:58:44 2014 -0700

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/43af2817/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala
index 197167e..0c97eb0 100644
--- a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala
+++ b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala
@@ -83,8 +83,7 @@ class UnionRDD[T: ClassTag](
 
   override def compute(s: Partition, context: TaskContext): Iterator[T] = {
     val part = s.asInstanceOf[UnionPartition[T]]
-    val parentRdd = dependencies(part.parentRddIndex).rdd.asInstanceOf[RDD[T]]
-    parentRdd.iterator(part.parentPartition, context)
+    parent[T](part.parentRddIndex).iterator(part.parentPartition, context)
   }
 
   override def getPreferredLocations(s: Partition): Seq[String] =


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message