hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Total input paths number and output
Date Sat, 02 Oct 2010 17:01:32 GMT
mapred.min.split.size and minimum map tasks properties of Hadoop MR also
control the splitting of input for map talks.

On Oct 2, 2010 10:28 PM, "Harsh J" <qwertymaniac@gmail.com> wrote:

Outputs are not dependent on number of inputs, but instead the number of
reducers (if MapReduce) or number of input splits if just plain Maps.

The number of splits is determined in most cases by the input file sizes and
the set HDFS block size factor (dfs.block.size) it was created under.

> On Oct 2, 2010 10:01 PM, "Shi Yu" <shiyu@uchicago.edu> wrote:
> Hi,
> I am running some cod...

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message