Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
From: Siddharth Ubale <siddharth.ubale@syncoms.com>
To: "user@spark.apache.org" <user@spark.apache.org>, "user@hadoop.apache.org"
	<user@hadoop.apache.org>
Subject: Container exited with a non-zero exit code 1-SparkJOb on YARN
Thread-Topic: Container exited with a non-zero exit code 1-SparkJOb on YARN
Thread-Index: AdFTfU9KGSOOJ+2XRfSlE7VaZPGRXw==
Date: Wed, 20 Jan 2016 12:29:47 +0000
Message-ID: 
 <AM4PR07MB1444DA9352EC65152A6B226BEFC20@AM4PR07MB1444.eurprd07.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: syncoms.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:23
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_AM4PR07MB1444DA9352EC65152A6B226BEFC20AM4PR07MB1444eurp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jan 2016 12:29:47.4174
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 49099e21-d9d9-45dc-b95b-f5f4afb3b3ff
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR07MB1441

--_000_AM4PR07MB1444DA9352EC65152A6B226BEFC20AM4PR07MB1444eurp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

I am running a Spark Job on the yarn cluster.
The spark job is a spark streaming application which is reading JSON from a=
 kafka topic , inserting the JSON values to hbase tables via Phoenix , ands=
 then sending out certain messages to a websocket if the JSON satisfies a c=
ertain criteria.

My cluster is a 3 node cluster with 24GB ram and 24 cores in total.

Now :
1. when I am submitting the job with 10GB memory, the application fails say=
ing memory is insufficient to run the job
2. The job is submitted with 6G ram. However, it does not run successfully =
always.Common issues faced :
                a. Container exited with a non-zero exit code 1 , and after=
 multiple such warning the job is finished.
                d. The failed job notifies that it was unable to find a fil=
e in HDFS which is something like _hadoop_conf_xxxxxx.zip

Can someone pls let me know why am I seeing the above 2 issues.

Thanks,
Siddharth Ubale,


--_000_AM4PR07MB1444DA9352EC65152A6B226BEFC20AM4PR07MB1444eurp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:#0563C1;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:#954F72;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">I am running a Spark Job on the yarn cluster.<o:p></=
o:p></p>
<p class=3D"MsoNormal">The spark job is a spark streaming application which=
 is reading JSON from a kafka topic , inserting the JSON values to hbase ta=
bles via Phoenix , ands then sending out certain messages to a websocket if=
 the JSON satisfies a certain criteria.
<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">My cluster is a 3 node cluster with 24GB ram and 24 =
cores in total.
<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Now :<o:p></o:p></p>
<p class=3D"MsoNormal">1. when I am submitting the job with 10GB memory, th=
e application fails saying memory is insufficient to run the job<o:p></o:p>=
</p>
<p class=3D"MsoNormal">2. The job is submitted with 6G ram. However, it doe=
s not run successfully always.Common issues faced :<o:p></o:p></p>
<p class=3D"MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; a. Container exited with a non-zero =
exit code 1 , and after multiple such warning the job is finished.<o:p></o:=
p></p>
<p class=3D"MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d. The failed job notifies that it w=
as unable to find a file in HDFS which is something like _<i>hadoop_conf</i=
>_xxxxxx.zip<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Can someone pls let me know why am I seeing the abov=
e 2 issues.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
<p class=3D"MsoNormal">Siddharth Ubale,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</body>
</html>

--_000_AM4PR07MB1444DA9352EC65152A6B226BEFC20AM4PR07MB1444eurp_--