hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kawa <kawa.a...@gmail.com>
Subject Re: The number of simultaneous map tasks is unexpected.
Date Tue, 08 Jul 2014 16:42:13 GMT
Is not your MapReduce AppMaster occupying one slot?

Sent from my iPhone

> On 8 jul 2014, at 13:01, Tomasz GuziaƂek <tomaszguzialek@gmail.com> wrote:
> 
> Hello all,
> 
> I am running a 4-nodes CDH5 cluster on Amazon EC2 . The instances used are m1.large,
so I have 4 cores (2 core x 2 unit) per node. My HBase table has 8 regions, so I expected
at least 8 (if not 16) mapper tasks to run simultaneously. However, only 7 are running and
1 is waiting for an empty slot. Why this surprising number came up? I have checked that the
regions are equally distributed on the region servers (2 per node).
> 
> My properties in the job:
> Configuration mapReduceConfiguration = HBaseConfiguration.create();
> mapReduceConfiguration.set("hbase.client.max.perregion.tasks", "4");
> mapReduceConfiguration.set("mapreduce.tasktracker.map.tasks.maximum", "16");
> 
> My properties in the CDH:
> yarn.scheduler.minimum-allocation-vcores = 1
> yarn.scheduler.maximum-allocation-vcores = 4
> 
> Do I miss some property? Please share your experience.
> 
> Best regards
> Tomasz

Mime
View raw message