Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <CANW4+F70bo5FAuPLCWqHV8VdD2F0CF8DeXCb6+5588AqO2QbJg@mail.gmail.com>
References: <CANW4+F70bo5FAuPLCWqHV8VdD2F0CF8DeXCb6+5588AqO2QbJg@mail.gmail.com>
From: Ajay Chander <itschevva@gmail.com>
Date: Mon, 7 Nov 2016 21:42:46 -0500
Message-ID: <CANW4+F4NAB-jDLiO+Qo-avvPdOundbX3+f6J1stpdpuhUyGdYg@mail.gmail.com>
Subject: Re: Access_Remote_Kerberized_Cluster_Through_Spark
To: user spark mailing list <user@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a113ec61e76b4110540c118ed
archived-at: Tue, 08 Nov 2016 02:43:00 -0000

--001a113ec61e76b4110540c118ed
Content-Type: text/plain; charset=UTF-8

Did anyone use
https://www.codatlas.com/github.com/apache/spark/HEAD/core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
to interact with secured Hadoop from Spark ?

Thanks,
Ajay

On Mon, Nov 7, 2016 at 4:37 PM, Ajay Chander <itschevva@gmail.com> wrote:

>
> Hi Everyone,
>
> I am trying to develop a simple codebase on my machine to read data from
> secured Hadoop cluster. We have a development cluster which is secured
> through Kerberos and I want to run a Spark job from my IntelliJ to read
> some sample data from the cluster. Has anyone done this before ? Can you
> point me to some sample examples?
>
> I understand that, if we want to talk to secured cluster, we need to have
> keytab and principle. I tried using it through UserGroupInformation.loginUserFromKeytab
> and SparkHadoopUtil.get.loginUserFromKeytab but so far no luck.
>
> I have been trying to do this from quite a while ago. Please let me know
> if you need more info. Thanks
>
> Regards,
> Ajay
>

--001a113ec61e76b4110540c118ed
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Did anyone use <a href=3D"https://www.codatlas.com/github.=
com/apache/spark/HEAD/core/src/main/scala/org/apache/spark/deploy/SparkHado=
opUtil.scala">https://www.codatlas.com/github.com/apache/spark/HEAD/core/sr=
c/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala</a> to interact =
with secured Hadoop from Spark ?<div><br></div><div>Thanks,</div><div>Ajay<=
/div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Mon=
, Nov 7, 2016 at 4:37 PM, Ajay Chander <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:itschevva@gmail.com" target=3D"_blank">itschevva@gmail.com</a>&gt;</spa=
n> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><br></div=
><div>Hi Everyone,</div><div><br></div><div>I am trying to develop a simple=
 codebase on my machine to read data from secured Hadoop cluster. We have a=
 development cluster which is secured through Kerberos and I want to run a =
Spark job from my IntelliJ to read some sample data from the cluster. Has a=
nyone done this before ? Can you point me to some sample examples?=C2=A0</d=
iv><div><br></div><div>I understand that, if we want to talk to secured clu=
ster, we need to have keytab and principle. I tried using it through <span =
style=3D"background-color:rgb(255,255,0)">UserGroupInformation.<wbr>loginUs=
erFromKeytab </span>and <span style=3D"background-color:rgb(255,255,0)">Spa=
rkHadoopUtil.get.<wbr>loginUserFromKeytab</span> but so far no luck.=C2=A0<=
/div><div><br></div><div>I have been trying to do this from quite a while a=
go. Please let me know if you need more info. Thanks</div><div><br></div><d=
iv>Regards,</div><div>Ajay</div></div>
</blockquote></div><br></div>

--001a113ec61e76b4110540c118ed--