hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin O'dell" <kevin.od...@cloudera.com>
Subject Re: Force the number of map tasks in MR?
Date Thu, 11 Oct 2012 20:36:39 GMT
J-M,

  It should be in the mapred-site.xml the values
are mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum.  This is the default in CDH4

<property>
  <name>mapreduce.tasktracker.map.tasks.maximum</name>
  <value>2</value>
  <description>The maximum number of map tasks that will be run
  simultaneously by a task tracker.
  </description>
</property>

<property>
  <name>mapreduce.tasktracker.reduce.tasks.maximum</name>
  <value>2</value>
  <description>The maximum number of reduce tasks that will be run
  simultaneously by a task tracker.
  </description>
</property>

This would explain why they are going 2 by 2.  Does this help?

On Thu, Oct 11, 2012 at 4:25 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> I don't know. I did not touched that. Where can I found this information?
>
> 2012/10/11 Kevin O'dell <kevin.odell@cloudera.com>:
> > What are you max tasks set to?
> >
> > On Thu, Oct 11, 2012 at 3:59 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> >> Hi,
> >>
> >> Is there a way to force the number of map tasks in a MR?
> >>
> >> I have a 25 regions table splitted over 6 nodes. But the MR is running
> >> the tasks only 2 by 2.
> >>
> >> Is there a way to force it to run one task on each regionserver
> >> serving at least one region? Why is the MR waiting for 2 taskes to
> >> complete before sending to the other tasks?
> >>
> >> I'm starting the MR with a caching of 100.
> >>
> >> I tried mapred.map.tasks and speculative=false with no success.
> >>
> >> Any idea how I can increase it this number of tasks?
> >>
> >> Thanks,
> >>
> >> JM
> >>
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
>



-- 
Kevin O'Dell
Customer Operations Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message