spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject spark git commit: [SPARK-7317] [Shuffle] Expose shuffle handle
Date Sat, 02 May 2015 04:23:50 GMT
Repository: spark
Updated Branches:
  refs/heads/master 38d4e9e44 -> b79aeb95b


[SPARK-7317] [Shuffle] Expose shuffle handle

Details in JIRA, in a nut-shell, all machinary for custom RDD's to leverage spark shuffle
directly (without exposing impl details of shuffle) exists - except for this small piece.

Exposing this will allow for custom dependencies to get a handle to ShuffleHandle - which
they can then leverage on reduce side.

Author: Mridul Muralidharan <mridulm@yahoo-inc.com>

Closes #5857 from mridulm/expose_shuffle_handle and squashes the following commits:

d8b6bd4 [Mridul Muralidharan] Expose ShuffleHandle


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b79aeb95
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b79aeb95
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b79aeb95

Branch: refs/heads/master
Commit: b79aeb95b45ab4ae811039d452cf028d7b844132
Parents: 38d4e9e
Author: Mridul Muralidharan <mridulm@yahoo-inc.com>
Authored: Fri May 1 21:23:42 2015 -0700
Committer: Reynold Xin <rxin@databricks.com>
Committed: Fri May 1 21:23:42 2015 -0700

----------------------------------------------------------------------
 .../src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b79aeb95/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala b/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
index 13c7115..e04c97f 100644
--- a/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
+++ b/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
@@ -17,9 +17,12 @@
 
 package org.apache.spark.shuffle
 
+import org.apache.spark.annotation.DeveloperApi
+
 /**
  * An opaque handle to a shuffle, used by a ShuffleManager to pass information about it to
tasks.
  *
  * @param shuffleId ID of the shuffle
  */
-private[spark] abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}
+@DeveloperApi
+abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message