spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject [3/3] git commit: Merge pull request #52 from harveyfeng/hadoop-closure
Date Sun, 13 Oct 2013 04:23:58 GMT
Merge pull request #52 from harveyfeng/hadoop-closure

Add an optional closure parameter to HadoopRDD instantiation to use when creating local JobConfs.

Having HadoopRDD accept this optional closure eliminates the need for the HadoopFileRDD added
earlier. It makes the HadoopRDD more general, in that the caller can specify any JobConf initialization
flow.


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/99796904
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/99796904
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/99796904

Branch: refs/heads/master
Commit: 99796904ae9d00405584ac518b6144973a876e3c
Parents: dca8009 6c32aab
Author: Reynold Xin <rxin@apache.org>
Authored: Sat Oct 12 21:23:26 2013 -0700
Committer: Reynold Xin <rxin@apache.org>
Committed: Sat Oct 12 21:23:26 2013 -0700

----------------------------------------------------------------------
 .../scala/org/apache/spark/SparkContext.scala   | 29 ++++-------
 .../scala/org/apache/spark/rdd/HadoopRDD.scala  | 52 ++++++--------------
 2 files changed, 26 insertions(+), 55 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/99796904/core/src/main/scala/org/apache/spark/SparkContext.scala
----------------------------------------------------------------------


Mime
View raw message