hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hari <harib...@gmail.com>
Subject YARN creates only 1 container
Date Tue, 27 May 2014 07:31:38 GMT
Hi,

When using YARN 2.2.0 version, only 1 container is created
for an application in the entire cluster.
The single container is created at an arbitrary node
for every run. This happens when running any application from
the examples jar (e.g., wordcount). Currently only one application is
run at a time. The input datasize is > 200GB.

I am setting custom values that affect concurrent container count.
These config parameters were mostly taken from:
http://blog.cloudera.com/blog/2014/04/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas/
These wasn't much description elsewhere on how the container count would be
decided.

The settings are:

mapred-site.xml
mapreduce.map.cpu.vcores = 1
mapreduce.reduce.cpu.vcores = 1
mapreduce.map.memory.mb = 1024
mapreduce.reduce.memory.mb = 1024
mapreduce.map.java.opts = -Xmx1024m
mapreduce.reduce.java.opts = -Xmx1024m

yarn.xml
yarn.nodemanager.resource.memory-mb = 65536
yarn.nodemanager.resource.cpu-vcores = 16

>From these settings, each node should be running 16 containers.

Let me know if there might be something else affecting the container
count.

thanks,
hari

Mime
View raw message