Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of jbishop.rwc@gmail.com
 designates 209.85.220.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAPQV63WU-jcenK5h_roECQtR9wxZfNQoYwV1LdFqahc8EzsZCg@mail.gmail.com>
References: 
 <CAPQV63UOLTMTPv88CVUqMKhc6zBbDJ=cngJtPgug_s3VsYAdug@mail.gmail.com>
	<CAGngS9fHkWbDHuMGEoKVd5PQbuEnH8BUdKGk7BxKoHTBS6dvWA@mail.gmail.com>
	<CAPQV63XHuOjePp78Rqj+aQ1T9zK_pFnnNbSmjbhkpQU0xgOVMQ@mail.gmail.com>
	<CAGngS9daxs=MxRvqHkhnQpUoPnOGTbn0sLBrsbS84BMz56Qdpw@mail.gmail.com>
	<CAPQV63WU-jcenK5h_roECQtR9wxZfNQoYwV1LdFqahc8EzsZCg@mail.gmail.com>
Date: Thu, 11 Oct 2012 13:50:44 -0700
Message-ID: 
 <CADfEU033N9ijUSjtdLBf4cVLoz87Mxw0ucDsS-zbZhL0xmjC7w@mail.gmail.com>
Subject: Re: Force the number of map tasks in MR?
From: Jonathan Bishop <jbishop.rwc@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=bcaec53aef50998b3704cbceb962

--bcaec53aef50998b3704cbceb962
Content-Type: text/plain; charset=ISO-8859-1

JM,

The number of map tasks will be limited by the number of input splits
available. Assuming you are reading files, that is.

Also, you need to reboot your cluster for those setting to take effect.

Hope this helps,

Jon Bishop

On Thu, Oct 11, 2012 at 1:44 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> But this is the limit per tasktracker, right?
>
> And I have 6 nodes, so 6 tasktrackers, which mean it should go up to 12
> tasks?
>
> Take a look at 2.7 here: http://wiki.apache.org/hadoop/FAQ
>
> I just tried with the setting below (changing 2 by 6) but I'm getting
> the same result.
>
> JM
>
> 2012/10/11 Kevin O'dell <kevin.odell@cloudera.com>:
> > J-M,
> >
> >   It should be in the mapred-site.xml the values
> > are mapred.tasktracker.map.tasks.maximum and
> > mapred.tasktracker.reduce.tasks.maximum.  This is the default in CDH4
> >
> > <property>
> >   <name>mapreduce.tasktracker.map.tasks.maximum</name>
> >   <value>2</value>
> >   <description>The maximum number of map tasks that will be run
> >   simultaneously by a task tracker.
> >   </description>
> > </property>
> >
> > <property>
> >   <name>mapreduce.tasktracker.reduce.tasks.maximum</name>
> >   <value>2</value>
> >   <description>The maximum number of reduce tasks that will be run
> >   simultaneously by a task tracker.
> >   </description>
> > </property>
> >
> > This would explain why they are going 2 by 2.  Does this help?
> >
> > On Thu, Oct 11, 2012 at 4:25 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> >> I don't know. I did not touched that. Where can I found this
> information?
> >>
> >> 2012/10/11 Kevin O'dell <kevin.odell@cloudera.com>:
> >> > What are you max tasks set to?
> >> >
> >> > On Thu, Oct 11, 2012 at 3:59 PM, Jean-Marc Spaggiari <
> >> > jean-marc@spaggiari.org> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> Is there a way to force the number of map tasks in a MR?
> >> >>
> >> >> I have a 25 regions table splitted over 6 nodes. But the MR is
> running
> >> >> the tasks only 2 by 2.
> >> >>
> >> >> Is there a way to force it to run one task on each regionserver
> >> >> serving at least one region? Why is the MR waiting for 2 taskes to
> >> >> complete before sending to the other tasks?
> >> >>
> >> >> I'm starting the MR with a caching of 100.
> >> >>
> >> >> I tried mapred.map.tasks and speculative=false with no success.
> >> >>
> >> >> Any idea how I can increase it this number of tasks?
> >> >>
> >> >> Thanks,
> >> >>
> >> >> JM
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Kevin O'Dell
> >> > Customer Operations Engineer, Cloudera
> >>
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
>

--bcaec53aef50998b3704cbceb962--