Return-Path: X-Original-To: apmail-spark-commits-archive@minotaur.apache.org Delivered-To: apmail-spark-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 653BC10962 for ; Sun, 13 Oct 2013 04:24:31 +0000 (UTC) Received: (qmail 43237 invoked by uid 500); 13 Oct 2013 04:24:26 -0000 Delivered-To: apmail-spark-commits-archive@spark.apache.org Received: (qmail 43125 invoked by uid 500); 13 Oct 2013 04:24:26 -0000 Mailing-List: contact commits-help@spark.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@spark.incubator.apache.org Delivered-To: mailing list commits@spark.incubator.apache.org Received: (qmail 43094 invoked by uid 99); 13 Oct 2013 04:24:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Oct 2013 04:24:18 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 13 Oct 2013 04:24:17 +0000 Received: (qmail 42942 invoked by uid 99); 13 Oct 2013 04:23:56 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Oct 2013 04:23:56 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 765A383856D; Sun, 13 Oct 2013 04:23:56 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rxin@apache.org To: commits@spark.incubator.apache.org Date: Sun, 13 Oct 2013 04:23:58 -0000 Message-Id: <2a8fbf31a524484c9d42912f7edcfc5c@git.apache.org> In-Reply-To: <27f5ab1594234c62a4dca76d1eed6165@git.apache.org> References: <27f5ab1594234c62a4dca76d1eed6165@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/3] git commit: Merge pull request #52 from harveyfeng/hadoop-closure X-Virus-Checked: Checked by ClamAV on apache.org Merge pull request #52 from harveyfeng/hadoop-closure Add an optional closure parameter to HadoopRDD instantiation to use when creating local JobConfs. Having HadoopRDD accept this optional closure eliminates the need for the HadoopFileRDD added earlier. It makes the HadoopRDD more general, in that the caller can specify any JobConf initialization flow. Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/99796904 Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/99796904 Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/99796904 Branch: refs/heads/master Commit: 99796904ae9d00405584ac518b6144973a876e3c Parents: dca8009 6c32aab Author: Reynold Xin Authored: Sat Oct 12 21:23:26 2013 -0700 Committer: Reynold Xin Committed: Sat Oct 12 21:23:26 2013 -0700 ---------------------------------------------------------------------- .../scala/org/apache/spark/SparkContext.scala | 29 ++++------- .../scala/org/apache/spark/rdd/HadoopRDD.scala | 52 ++++++-------------- 2 files changed, 26 insertions(+), 55 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/99796904/core/src/main/scala/org/apache/spark/SparkContext.scala ----------------------------------------------------------------------