hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Force the number of map tasks in MR?
Date Thu, 11 Oct 2012 20:44:59 GMT
But this is the limit per tasktracker, right?

And I have 6 nodes, so 6 tasktrackers, which mean it should go up to 12 tasks?

Take a look at 2.7 here: http://wiki.apache.org/hadoop/FAQ

I just tried with the setting below (changing 2 by 6) but I'm getting
the same result.

JM

2012/10/11 Kevin O'dell <kevin.odell@cloudera.com>:
> J-M,
>
>   It should be in the mapred-site.xml the values
> are mapred.tasktracker.map.tasks.maximum and
> mapred.tasktracker.reduce.tasks.maximum.  This is the default in CDH4
>
> <property>
>   <name>mapreduce.tasktracker.map.tasks.maximum</name>
>   <value>2</value>
>   <description>The maximum number of map tasks that will be run
>   simultaneously by a task tracker.
>   </description>
> </property>
>
> <property>
>   <name>mapreduce.tasktracker.reduce.tasks.maximum</name>
>   <value>2</value>
>   <description>The maximum number of reduce tasks that will be run
>   simultaneously by a task tracker.
>   </description>
> </property>
>
> This would explain why they are going 2 by 2.  Does this help?
>
> On Thu, Oct 11, 2012 at 4:25 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> I don't know. I did not touched that. Where can I found this information?
>>
>> 2012/10/11 Kevin O'dell <kevin.odell@cloudera.com>:
>> > What are you max tasks set to?
>> >
>> > On Thu, Oct 11, 2012 at 3:59 PM, Jean-Marc Spaggiari <
>> > jean-marc@spaggiari.org> wrote:
>> >
>> >> Hi,
>> >>
>> >> Is there a way to force the number of map tasks in a MR?
>> >>
>> >> I have a 25 regions table splitted over 6 nodes. But the MR is running
>> >> the tasks only 2 by 2.
>> >>
>> >> Is there a way to force it to run one task on each regionserver
>> >> serving at least one region? Why is the MR waiting for 2 taskes to
>> >> complete before sending to the other tasks?
>> >>
>> >> I'm starting the MR with a caching of 100.
>> >>
>> >> I tried mapred.map.tasks and speculative=false with no success.
>> >>
>> >> Any idea how I can increase it this number of tasks?
>> >>
>> >> Thanks,
>> >>
>> >> JM
>> >>
>> >
>> >
>> >
>> > --
>> > Kevin O'Dell
>> > Customer Operations Engineer, Cloudera
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera

Mime
View raw message