hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomasz GuziaƂek <tomaszguzia...@gmail.com>
Subject The number of simultaneous map tasks is unexpected.
Date Tue, 08 Jul 2014 11:01:58 GMT
Hello all,

I am running a 4-nodes CDH5 cluster on Amazon EC2 . The instances used are
m1.large, so I have 4 cores (2 core x 2 unit) per node. My HBase table has
8 regions, so I expected at least 8 (if not 16) mapper tasks to run
simultaneously. However, only 7 are running and 1 is waiting for an empty
slot. Why this surprising number came up? I have checked that the regions
are equally distributed on the region servers (2 per node).

My properties in the job:
Configuration mapReduceConfiguration = HBaseConfiguration.create();
mapReduceConfiguration.set("hbase.client.max.perregion.tasks", "4");
mapReduceConfiguration.set("mapreduce.tasktracker.map.tasks.maximum", "16");

My properties in the CDH:
yarn.scheduler.minimum-allocation-vcores = 1
yarn.scheduler.maximum-allocation-vcores = 4

Do I miss some property? Please share your experience.

Best regards

View raw message