Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of write2kishore@gmail.com
 designates 209.85.223.169 as permitted sender)
MIME-Version: 1.0
Date: Fri, 22 Mar 2013 15:56:54 +0530
Message-ID: 
 <CAHg+sbO5x+Ux5sqbtD+Rh33zTF4JGhBTooyuvWAHk9CX7sWZ-w@mail.gmail.com>
Subject: Application Master getting killed randomly reporting excess usage of
 memory
From: Krishna Kishore Bonagiri <write2kishore@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae9340bffedbc4e04d880e48a

--14dae9340bffedbc4e04d880e48a
Content-Type: text/plain; charset=ISO-8859-1

Hi,

  I am running a date command using the Distributed Shell example in a loop
of 500 times. It ran successfully all the times except one time where it
gave the following error.

2013-03-22 04:33:25,280 INFO  [main] distributedshell.Client
(Client.java:monitorApplication(605)) - Got application report from ASM
for, appId=222, clientToken=null, appDiagnostics=Application
application_1363938200742_0222 failed 1 times due to AM Container for
appattempt_1363938200742_0222_000001 exited with  exitCode: 143 due to:
Container [pid=21141,containerID=container_1363938200742_0222_01_000001] is
running beyond virtual memory limits. Current usage: 47.3 Mb of 128 Mb
physical memory used; 611.6 Mb of 268.8 Mb virtual memory used. Killing
container.
Dump of the process-tree for container_1363938200742_0222_01_000001 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 21147 21141 21141 21141 (java) 244 12 532643840 11802
/home_/dsadm/yarn/jdk//bin/java -Xmx128m
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster
--container_memory 10 --num_containers 2 --priority 0 --shell_command date
        |- 21141 8433 21141 21141 (bash) 0 0 108642304 298 /bin/bash -c
/home_/dsadm/yarn/jdk//bin/java -Xmx128m
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster
--container_memory 10 --num_containers 2 --priority 0 --shell_command date
1>/tmp/logs/application_1363938200742_0222/container_1363938200742_0222_01_000001/AppMaster.stdout
2>/tmp/logs/application_1363938200742_0222/container_1363938200742_0222_01_000001/AppMaster.stderr


  Any ideas if it is a known issue? I am using the latest version of
hadoop, i.e. hadoop-2.0.3-alpha.

Thanks,
Kishore

--14dae9340bffedbc4e04d880e48a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div><br><div style>=A0 I am running a date command usi=
ng the Distributed Shell example in a loop of 500 times. It ran successfull=
y all the times except one time where it gave the following error.</div><di=
v style>
<br></div><div style><div>2013-03-22 04:33:25,280 INFO =A0[main] distribute=
dshell.Client (Client.java:monitorApplication(605)) - Got application repor=
t from ASM for, appId=3D222, clientToken=3Dnull, appDiagnostics=3DApplicati=
on application_1363938200742_0222 failed 1 times due to AM Container for ap=
pattempt_1363938200742_0222_000001 exited with =A0exitCode: 143 due to: Con=
tainer [pid=3D21141,containerID=3Dcontainer_1363938200742_0222_01_000001] i=
s running beyond virtual memory limits. Current usage: 47.3 Mb of 128 Mb ph=
ysical memory used; 611.6 Mb of 268.8 Mb virtual memory used. Killing conta=
iner.</div>
<div>Dump of the process-tree for container_1363938200742_0222_01_000001 :<=
/div><div>=A0 =A0 =A0 =A0 |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME=
(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD=
_LINE</div>
<div>=A0 =A0 =A0 =A0 |- 21147 21141 21141 21141 (java) 244 12 532643840 118=
02 /home_/dsadm/yarn/jdk//bin/java -Xmx128m org.apache.hadoop.yarn.applicat=
ions.distributedshell.ApplicationMaster --container_memory 10 --num_contain=
ers 2 --priority 0 --shell_command date</div>
<div>=A0 =A0 =A0 =A0 |- 21141 8433 21141 21141 (bash) 0 0 108642304 298 /bi=
n/bash -c /home_/dsadm/yarn/jdk//bin/java -Xmx128m org.apache.hadoop.yarn.a=
pplications.distributedshell.ApplicationMaster --container_memory 10 --num_=
containers 2 --priority 0 --shell_command date 1&gt;/tmp/logs/application_1=
363938200742_0222/container_1363938200742_0222_01_000001/AppMaster.stdout 2=
&gt;/tmp/logs/application_1363938200742_0222/container_1363938200742_0222_0=
1_000001/AppMaster.stderr</div>
<div><br></div><div><br></div><div style>=A0 Any ideas if it is a known iss=
ue? I am using the latest version of hadoop, i.e. hadoop-2.0.3-alpha.</div>=
<div style><br></div><div style>Thanks,</div><div style>Kishore</div></div>
</div></div>

--14dae9340bffedbc4e04d880e48a--