Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of tesmai4@gmail.com designates
 209.85.215.52 as permitted sender)
MIME-Version: 1.0
From: "tesmai4@gmail.com" <tesmai4@gmail.com>
Date: Fri, 20 Feb 2015 17:49:27 +0000
Message-ID: 
 <CALD6Y59gjGcE2aT2eL20oZ06_XjctJfV1NrO2aV=bWzbKB91yg@mail.gmail.com>
Subject: Fwd: YARN container lauch failed exception and mapred-site.xml
 configuration
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a1133afa0b7a423050f88adea

--001a1133afa0b7a423050f88adea
Content-Type: text/plain; charset=UTF-8

I have 7 nodes in my Hadoop cluster [8GB RAM and 4VCPUs to each nodes], 1
Namenode + 6 datanodes.

I followed the link o horton works [
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.6.0/bk_installing_manually_book/content/rpm-chap1-11.html]
and made  calculation according to the hardware configruation on my nodes
and have added the update mapred-site and yarn-site.xml files in my
question. Still my application is crashing with the same exection

My mapreduce application has 34 input splits with a block size of 128MB.

**mapred-site.xml** has the  following properties:

    mapreduce.framework.name  = yarn
    mapred.child.java.opts    = -Xmx2048m
    mapreduce.map.memory.mb   = 4096
    mapreduce.map.java.opts   = -Xmx2048m

**yarn-site.xml** has the  following properties:

    yarn.resourcemanager.hostname        = hadoop-master
    yarn.nodemanager.aux-services        = mapreduce_shuffle
    yarn.nodemanager.resource.memory-mb  = 6144
    yarn.scheduler.minimum-allocation-mb = 2048
    yarn.scheduler.maximum-allocation-mb = 6144


 Exception from container-launch: ExitCodeException exitCode=134:
/bin/bash: line 1:  3876 Aborted  (core dumped)
/usr/lib/jvm/java-7-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true
-Dhadoop.metrics.log.level=WARN -Xmx8192m
-Djava.io.tmpdir=/tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1424264025191_0002/container_1424264025191_0002_01_000011/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir=/home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011
    -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
org.apache.hadoop.mapred.YarnChild 192.168.0.12 50842
attempt_1424264025191_0002_m_000005_0 11 >

/home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011/stdout
2>

/home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011/stderr


How can avoid this?any help is appreciated

Is there an option to restrict number of containers on hadoop ndoes?

--001a1133afa0b7a423050f88adea
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_quote"><br><div dir=3D"ltr"><div class=
=3D"gmail_quote"><div dir=3D"ltr"><div>I have 7 nodes in my Hadoop cluster =
[8GB RAM and 4VCPUs to each nodes], 1 Namenode + 6 datanodes.</div><div><br=
></div><div>I followed the link o horton works [<a href=3D"http://docs.hort=
onworks.com/HDPDocuments/HDP2/HDP-2.0.6.0/bk_installing_manually_book/conte=
nt/rpm-chap1-11.html">http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0=
.6.0/bk_installing_manually_book/content/rpm-chap1-11.html</a>] and made =
=C2=A0calculation according to the hardware configruation on my nodes and h=
ave added the update mapred-site and yarn-site.xml files in my question. St=
ill my application is crashing with the same exection</div><div>=C2=A0</div=
><div>My mapreduce application has 34 input splits with a block size of 128=
MB.</div><div><br></div><div>**mapred-site.xml** has the =C2=A0following pr=
operties:</div><div><br></div><div>=C2=A0 =C2=A0 <a href=3D"http://mapreduc=
e.framework.name" target=3D"_blank">mapreduce.framework.name</a> =C2=A0=3D =
yarn</div><div>=C2=A0 =C2=A0 mapred.child.java.opts =C2=A0 =C2=A0=3D -Xmx20=
48m</div><div>=C2=A0 =C2=A0 mapreduce.map.memory.mb =C2=A0 =3D 4096</div><d=
iv>=C2=A0 =C2=A0 mapreduce.map.java.opts =C2=A0 =3D -Xmx2048m</div><div><br=
></div><div>**yarn-site.xml** has the =C2=A0following properties:</div><div=
><br></div><div>=C2=A0 =C2=A0 yarn.resourcemanager.hostname =C2=A0 =C2=A0 =
=C2=A0 =C2=A0=3D hadoop-master</div><div>=C2=A0 =C2=A0 yarn.nodemanager.aux=
-services =C2=A0 =C2=A0 =C2=A0 =C2=A0=3D mapreduce_shuffle</div><div>=C2=A0=
 =C2=A0 yarn.nodemanager.resource.memory-mb =C2=A0=3D 6144</div><div>=C2=A0=
 =C2=A0 yarn.scheduler.minimum-allocation-mb =3D 2048</div><div>=C2=A0 =C2=
=A0 yarn.scheduler.maximum-allocation-mb =3D 6144</div><div><br></div><div>=
<br></div><div>=C2=A0Exception from container-launch: ExitCodeException exi=
tCode=3D134: /bin/bash: line 1: =C2=A03876 Aborted =C2=A0(core dumped) /usr=
/lib/jvm/java-7-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=3Dtrue -D=
hadoop.metrics.log.level=3DWARN -Xmx8192m -Djava.io.tmpdir=3D/tmp/hadoop-ub=
untu/nm-local-dir/usercache/ubuntu/appcache/application_1424264025191_0002/=
container_1424264025191_0002_01_000011/tmp -Dlog4j.configuration=3Dcontaine=
r-log4j.properties -Dyarn.app.container.log.dir=3D/home/ubuntu/hadoop/logs/=
userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000=
011<br></div><div>=C2=A0 =C2=A0 -Dyarn.app.container.log.filesize=3D0 -Dhad=
oop.root.logger=3DINFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.0.12 =
50842 attempt_1424264025191_0002_m_000005_0 11 &gt;=C2=A0</div><div>=C2=A0 =
=C2=A0 /home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/con=
tainer_1424264025191_0002_01_000011/stdout 2&gt;=C2=A0</div><div>=C2=A0 =C2=
=A0 /home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/contai=
ner_1424264025191_0002_01_000011/stderr</div><div><br></div><div><br></div>=
<div>How can avoid this?any help is appreciated=C2=A0</div><div><br></div><=
div>Is there an option to restrict number of containers on hadoop ndoes?</d=
iv><div><br></div></div>
</div><br></div>
</div><br></div>

--001a1133afa0b7a423050f88adea--