Return-Path: X-Original-To: apmail-spark-commits-archive@minotaur.apache.org Delivered-To: apmail-spark-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7DCA510FA2 for ; Thu, 23 Jan 2014 08:19:36 +0000 (UTC) Received: (qmail 69185 invoked by uid 500); 23 Jan 2014 08:19:35 -0000 Delivered-To: apmail-spark-commits-archive@spark.apache.org Received: (qmail 69159 invoked by uid 500); 23 Jan 2014 08:19:34 -0000 Mailing-List: contact commits-help@spark.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@spark.incubator.apache.org Delivered-To: mailing list commits@spark.incubator.apache.org Received: (qmail 69076 invoked by uid 99); 23 Jan 2014 08:19:28 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jan 2014 08:19:28 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 23 Jan 2014 08:19:25 +0000 Received: (qmail 69001 invoked by uid 99); 23 Jan 2014 08:19:02 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jan 2014 08:19:02 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id AB9098AC64C; Thu, 23 Jan 2014 08:19:02 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rxin@apache.org To: commits@spark.incubator.apache.org Date: Thu, 23 Jan 2014 08:19:04 -0000 Message-Id: In-Reply-To: <97607cd7013a42698b40ebe3544a78aa@git.apache.org> References: <97607cd7013a42698b40ebe3544a78aa@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/6] git commit: Add comment explaining collectPartitions's use X-Virus-Checked: Checked by ClamAV on apache.org Add comment explaining collectPartitions's use Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/3ef68e49 Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/3ef68e49 Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/3ef68e49 Branch: refs/heads/branch-0.8 Commit: 3ef68e49ef7c5548e2978beb7b2f903f00b50d95 Parents: 91e6e5b Author: Shivaram Venkataraman Authored: Thu Dec 19 11:49:17 2013 -0800 Committer: Shivaram Venkataraman Committed: Thu Jan 16 19:21:18 2014 -0800 ---------------------------------------------------------------------- core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/3ef68e49/core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala b/core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala index f3dd013..2283a8e 100644 --- a/core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala +++ b/core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala @@ -250,6 +250,8 @@ trait JavaRDDLike[T, This <: JavaRDDLike[T, This]] extends Serializable { * Return an array that contains all of the elements in a specific partition of this RDD. */ def collectPartitions(partitionIds: Array[Int]): Array[JList[T]] = { + // This is useful for implementing `take` from other language frontends + // like Python where the data is serialized. import scala.collection.JavaConversions._ val res = context.runJob(rdd, (it: Iterator[T]) => it.toArray, partitionIds, true) res.map(x => new java.util.ArrayList(x.toSeq)).toArray