Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
From: Saurav Sinha <sauravsinha76@gmail.com>
Date: Wed, 15 Mar 2017 16:06:47 +0530
Message-ID: <CAAFfpkHa0tYbToW+bNqCRYVoHwgem6ZGy9hcdWw=ucNQ7zoGtQ@mail.gmail.com>
Subject: Setting spark.yarn.stagingDir in 1.6
To: user <user@spark.apache.org>, vikash.pareek@infoobjects.com
Content-Type: multipart/alternative; boundary=001a114691e2886431054ac28566
archived-at: Wed, 15 Mar 2017 11:29:12 -0000

--001a114691e2886431054ac28566
Content-Type: text/plain; charset=UTF-8

Hi Users,


I am running spark job in yarn.

I want to set staging directory to some other location which is by default
hdfs://host:port/home/$User/

In spark 2.0.0, it can be done by setting spark.yarn.stagingDir.

But in production, we have spark 1.6. Can anyone please suggest how it can
be done in spark 1.6.

-- 
Thanks and Regards,

Saurav Sinha

Contact: 9742879062

--001a114691e2886431054ac28566
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Users,<div><br></div><div><br></div><div>I am running=
=C2=A0spark job in yarn.</div><div><br></div><div>I want to set staging dir=
ectory to some other location which is by default hdfs://host:port/home/$Us=
er/</div><div><br></div><div>In spark 2.0.0, it can be done by setting spar=
k.yarn.stagingDir.</div><div><br></div><div>But in production, we have spar=
k 1.6. Can anyone please=C2=A0suggest how it can be done in spark 1.6. =C2=
=A0</div><div><div><br></div>-- <br><div class=3D"gmail_signature"><div dir=
=3D"ltr">Thanks and Regards,<div><br></div><div>Saurav Sinha</div><div><br>=
</div><div>Contact: 9742879062</div></div></div>
</div></div>

--001a114691e2886431054ac28566--