hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels Basjes <Ni...@basjes.nl>
Subject Re: Not able to run more than one map task
Date Fri, 10 Apr 2015 23:01:18 GMT
Just curious: what is the input for your job ? If it is a single gzipped
file then that is the cause of getting exactly 1 mapper.

Niels

On Fri, Apr 10, 2015, 09:21 Amit Kumar <amitiitd@msn.com> wrote:

> Thanks a lot Harsha for replying
>
> This problem has waster at least last one week.
>
> We tried what you suggested. Could you please take a look at the
> configuration and suggest if we missed c?
>
> System RAM : 8GB
> CPU : 4 threads each with 2 cores.
> # Disks : 1
>
> MR2:
>
> mapreduce.map.memory.mb : 512
> mapreduce.tasktracker.map.tasks.maximum : 4
>
> Yarn:
>
> yarn.app.mapreduce.am.resource.mb : 512
> yarn.nodemanager.resource.cpu-vcores : 4
> yarn.scheduler.minimum-allocation-mb : 512
> yarn.nodemanager.resource.memory-mb : 5080
>
>
> Regards,
> Amit
>
>
>
> > From: harsh@cloudera.com
> > Date: Fri, 10 Apr 2015 10:20:24 +0530
> > Subject: Re: Not able to run more than one map task
> > To: user@hadoop.apache.org
>
> >
> > You are likely memory/vcore starved in the NM's configs. Increase your
> > yarn.nodemanager.resource.memory-mb and
> > yarn.nodemanager.resource.cpu-vcores configs, or consider lowering the
> > MR job memory request values to gain more parallelism.
> >
> > On Thu, Apr 9, 2015 at 5:05 PM, Amit Kumar <amitiitd@msn.com> wrote:
> > > Hi All,
> > >
> > > We recently started working on Hadoop. We have setup the hadoop in
> pseduo
> > > distribution mode along with oozie.
> > >
> > > Every developer has set it up on his laptop. The problem is that we
> are not
> > > able to run more than one map task concurrently on our laptops.
> Resource
> > > manager is not allowing more than one task on our machine.
> > >
> > > My task gets completed if I submit it without Oozie. Oozie requires
> one map
> > > task for its own functioning. Actual task that oozie submit does not
> start.
> > >
> > > Here is my configuration
> > >
> > > -- Hadoop setup in Pseudo distribution mode
> > > -- Hadoop Version - 2.6
> > > -- Oozie Version - 4.0.1
> > >
> > > Regards,
> > > Amit
> >
> >
> >
> > --
> > Harsh J
>

Mime
View raw message