hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject MapReduce Integration tests for test FileSystems
Date Sat, 13 Apr 2013 20:46:20 GMT
MapReduce has some very demanding file operations at at job submission
time, and is an important integration test for any hadoop compliant
FileSystem.

How in the hadoop HDFS source tree are mapreduce integration tests
implemented (also I'm interested in how S3, Swift, and other Hadoop
filesystem implementations) implement integration tests... if any clues.  I
haven't seen much in this way in the source code.

To distill my quesiton: how are new HDFS patches tested against the
MapReduce job flow --- ? --- are there some standalone vanilla MapReduce
jobs (Sorting, WordCount, etc..., that run as part of the HDFS build or is
there an HDFS-Mapreduce integration repository ?)


-- 
Jay Vyas
http://jayunit100.blogspot.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message