hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shekhar Sharma <shekhar2...@gmail.com>
Subject Re: 2 Map tasks running for a small input file
Date Thu, 26 Sep 2013 11:19:32 GMT
Number of map tasks on a mapreduce job doesnt depend on this
property..it depends on the number of input splits...( or equal to
number blocks if input split size = block size)

1. What is the input format you are using? if yes what is the value of
N, you are using?

2. WHat is the propety mapred.min.split.size? have you changed to
something else or is it default which is 1?




Regards,
Som Shekhar Sharma
+91-8197243810


On Thu, Sep 26, 2013 at 4:39 PM, Viji R <viji@cloudera.com> wrote:
> Hi,
>
> Default number of map tasks is 2. You can set mapred.map.tasks to 1 to
> avoid this.
>
> Regards,
> Viji
>
> On Thu, Sep 26, 2013 at 4:28 PM, Sai Sai <saigraph@yahoo.in> wrote:
>> Hi
>> Here is the input file for the wordcount job:
>> ******************
>> Hi This is a simple test.
>> Hi Hadoop how r u.
>> Hello Hello.
>> Hi Hi.
>> Hadoop Hadoop Welcome.
>> ******************
>>
>> After running the wordcount successfully
>> here r the counters info:
>>
>> ***************
>> Job Counters SLOTS_MILLIS_MAPS 0 0 8,386
>> Launched reduce tasks 0 0 1
>> Total time spent by all reduces waiting after reserving slots (ms) 0 0 0
>> Total time spent by all maps waiting after reserving slots (ms) 0 0 0
>> Launched map tasks 0 0 2
>> Data-local map tasks 0 0 2
>> SLOTS_MILLIS_REDUCES 0 0 9,199
>> ***************
>> My question why r there 2 launched map tasks when i have only a small file.
>> Per my understanding it is only 1 block.
>> and should be only 1 split.
>> Then for each line a map computation should occur
>> but it shows 2 map tasks.
>> Please let me know.
>> Thanks
>> Sai
>>

Mime
View raw message