Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4B56118572 for ; Tue, 27 Oct 2015 01:51:16 +0000 (UTC) Received: (qmail 68498 invoked by uid 500); 27 Oct 2015 01:51:05 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 68347 invoked by uid 500); 27 Oct 2015 01:51:04 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 68333 invoked by uid 99); 27 Oct 2015 01:51:04 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Oct 2015 01:51:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D7316C2B19 for ; Tue, 27 Oct 2015 01:51:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.011 X-Spam-Level: *** X-Spam-Status: No, score=3.011 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, RP_MATCHES_RCVD=-0.001, T_MIME_MALF=0.01, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id K_1IHM8rHKdD for ; Tue, 27 Oct 2015 01:50:52 +0000 (UTC) Received: from SNT004-OMC1S15.hotmail.com (snt004-omc1s15.hotmail.com [65.55.90.26]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 13BD842BB4 for ; Tue, 27 Oct 2015 01:50:52 +0000 (UTC) Received: from SNT405-EAS423 ([65.55.90.7]) by SNT004-OMC1S15.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Mon, 26 Oct 2015 18:50:45 -0700 X-TMN: [/tVMXrg333JhSeBkjzxkERsEUtPOW8Xl] X-Originating-Email: [jingfei_hu@hotmail.com] Message-ID: From: Jingfei Hu To: CC: References: ,<47A75D61-66EC-41F4-8508-E17EFF7694A3@hortonworks.com>,,,, In-Reply-To: Subject: RE: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Date: Tue, 27 Oct 2015 09:50:42 +0800 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0207_01D1109C.F1CA3AC0" X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQABAgME+tdqnJ7ee/RGdG7HsFs3SwDdycCSACiys+0A+FpskwDlcs2cAKkfu6cAhQNtPKH+G8XA Content-Language: en-us X-OriginalArrivalTime: 27 Oct 2015 01:50:45.0366 (UTC) FILETIME=[E52F2160:01D11059] ------=_NextPart_000_0207_01D1109C.F1CA3AC0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit HI Gaurav, I would appreciate that if you can give me a little more steps to troubleshoot where I am doing wrong. Thank you very much. Jingfei From: Jingfei Hu [mailto:jingfei_hu@hotmail.com] Sent: Thursday, October 22, 2015 11:20 PM To: user@hadoop.apache.org Subject: RE: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Hi Gaurav, ON which machine I should install the Kerberos client libraries? On which machine I execute the kinit command? Can't users just provide a username and password whenever He/She needs access a hdfs using webhdfs? In most cases, people have no or little knowledge about the kinit command. Is my understanding right? Thanks, Jingfei _____ Date: Thu, 22 Oct 2015 17:33:14 +0530 Subject: Re: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) From: gauravkumar37@gmail.com To: user@hadoop.apache.org Hi, Assuming you have Kerberos client libraries installed on your machine and that you have the correct configuration to point to the Kerberos KDC. You'll need to first do a kinit to authenticate to the Kerberos KDC and then when you do a curl you'll no longer see the error. At any time you can check your active tokens by executing klist -e Best Regards, Gaurav Kumar Big Data . Data Science . Photography . Music +91 9953294125 On Thu, Oct 22, 2015 at 1:29 PM, Jingfei Hu > wrote: Hi Anu, Thanks for the reply. Here is what I am doing in my client machine. C:\Users\adms>curl -i --negotiate -u: "http://xxx.cloudapp.net:50070/webhdfs/v1?op=gethomedirectory" HTTP/1.1 401 Authentication required Cache-Control: must-revalidate,no-cache,no-store Date: Thu, 22 Oct 2015 07:38:52 GMT Pragma: no-cache Date: Thu, 22 Oct 2015 07:38:52 GMT Pragma: no-cache Content-Type: text/html; charset=iso-8859-1 WWW-Authenticate: Negotiate Set-Cookie: hadoop.auth=; Path=/; Expires=Thu, 01-Jan-1970 00:00:00 GMT; HttpOnly Content-Length: 1403 Server: Jetty(6.1.26.hwx) HTTP/1.1 403 GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Cache-Control: must-revalidate,no-cache,no-store Date: Thu, 22 Oct 2015 07:38:52 GMT Pragma: no-cache Date: Thu, 22 Oct 2015 07:38:52 GMT Pragma: no-cache Content-Type: text/html; charset=iso-8859-1 Set-Cookie: hadoop.auth=; Path=/; Expires=Thu, 01-Jan-1970 00:00:00 GMT; HttpOnly Content-Length: 1561 Server: Jetty(6.1.26.hwx) Error 403 GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)

HTTP ERROR 403

Problem accessing /webhdfs/v1. Reason: 

    GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos credentails)

Powered by Jetty://
As for the command line shown in the url . Here is what I am trying on the KDC machine. kinit admin/admin After entering the password, it exited without problems curl -i --negotiate -u admin http://sandbox.hortonworks.com:50070/webhdfs/v1?op=gethomedirectory and enter the password for the subsequent prompt. It also returns ok. I can see the response's content is {"Path":"/user/admin"} However if I change the command line to be curl -i --negotiate -u admin http://xxx.cloudapp.net:50070/webhdfs/v1?op=gethomedirectory, it failed with Error 401 Authentication required

HTTP ERROR 401

Problem accessing /webhdfs/v1. Reason: 

    Authentication required

Powered by Jetty://
Any ideas? Thanks, Jingfei From: Anu Engineer [mailto:aengineer@hortonworks.com ] Sent: Thursday, October 22, 2015 12:16 PM To: user@hadoop.apache.org Subject: Re: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Did you try out with curl and it did not work at all ?. It is slightly easier to debug with command line than a browser like IE. https://hadoop.apache.org/docs/r2.6.0/hadoop-auth/Examples.html Please try getting a ticket and try to connect the cluster using it and if does not work, could you please post your traces. This will allow us rule out the issue that you might not have a valid kerberos ticket in the first place. Thanks Anu From: Jingfei Hu Reply-To: "user@hadoop.apache.org " Date: Wednesday, October 21, 2015 at 7:30 PM To: "user@hadoop.apache.org " Subject: RE: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) The links Anu provides seems to be no much helpful for my problem. I would give more context here. I am using HDP on Azure. Its version is 2.3. After several attempts, I'm finally able to enable Kerberos for my HDP deployment. Now I am trying to use webhdfs to access hdfs files in IE. http://xxx.cloudapp.net:50070/webhdfs/v1?op=gethomedirectory. Then IE pops up a windows to ask for user name and password. Then I enter the admin principal in KDC domain style (admin/admin@DOMAIN.com ) along with the correct password. However it gives me GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentials) Before enabling Kerberos, I can access this without entering anything because its security is off. Now how am i gonna do to make it work with Kerberos enabled? Jingfei From: Anu Engineer [mailto:aengineer@hortonworks.com] Sent: Wednesday, October 21, 2015 11:42 PM To: user@hadoop.apache.org Subject: Re: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) This usually means that you don't have valid Kerberos credentials. Please look at the authentication section at this URL https://hadoop.apache.org/docs/r1.0.4/webhdfs.html or here is another example. http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-3- 2/CDH4-Security-Guide/cdh4sg_topic_7_2.html Thanks Anu From: Jingfei Hu Reply-To: "user@hadoop.apache.org " Date: Tuesday, October 20, 2015 at 10:45 PM To: "user@hadoop.apache.org " Subject: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) Hi team, Is there anyone who knows what this exception means? What should I do to fix this? I am trying to access http://xxx.cloudapp.net:50070/webhdfs/v1/?op=liststatus I just enabled the Kerberos for the HDFS deployment. Thanks, Jingfei ------=_NextPart_000_0207_01D1109C.F1CA3AC0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

HI Gaurav,

I would appreciate that if you can give me a little more steps to = troubleshoot where I am doing wrong. Thank you very = much.

 

Jingfei

 

From:<= /b> = Jingfei Hu [mailto:jingfei_hu@hotmail.com]
Sent: Thursday, = October 22, 2015 11:20 PM
To: = user@hadoop.apache.org
Subject: RE: GSSException: No valid = credentials provided (Mechanism level: Failed to find any Kerberos = credentails)

 

Hi = Gaurav,

 

ON which machine I should = install the Kerberos client = libraries?

 

On which machine I execute = the kinit command?

 

Can't users just provide a = username and password whenever He/She needs access a hdfs = using webhdfs? In most cases, people have no or little knowledge = about the kinit command.  Is my understanding = right?

 

Thanks,

<= /div>

Jingfei

<= /div>

Date: Thu, 22 Oct 2015 = 17:33:14 +0530
Subject: Re: GSSException: No valid credentials = provided (Mechanism level: Failed to find any Kerberos = credentails)
From: gauravkumar37@gmail.com
To= : user@hadoop.apache.org

Hi,

Assuming you have Kerberos = client libraries installed on your machine and that you have the correct = configuration to point to the Kerberos = KDC.

You'll need to first do a = kinit to authenticate to the Kerberos KDC and then when you do a curl = you'll no longer see the error.

At any time you can check = your active tokens by executing klist = -e



Best Regards,
Gaurav = Kumar
Big Data • Data Science • Photography • = Music
+91 = 9953294125

 

On Thu, Oct 22, 2015 at 1:29 = PM, Jingfei Hu <jingfei_hu@hotmail.com> = wrote:

Hi = Anu,

Thanks for the = reply. Here is what I am doing in my client machine.

C:\Users\adms>= ;curl -i --negotiate -u: "http://xxx.cloudapp.net:50070/webhdfs/v1?op=3Dgethomedi= rectory"

HTTP/1.1 401 = Authentication required

Cache-Control: = must-revalidate,no-cache,no-store

Date: Thu, 22 = Oct 2015 07:38:52 GMT

Pragma: = no-cache

Date: Thu, 22 = Oct 2015 07:38:52 GMT

Pragma: = no-cache

Content-Type: = text/html; charset=3Diso-8859-1

WWW-Authenticate= : Negotiate

Set-Cookie: = hadoop.auth=3D; Path=3D/; Expires=3DThu, 01-Jan-1970 00:00:00 GMT; = HttpOnly

Content-Length: = 1403

Server: = Jetty(6.1.26.hwx)

 

HTTP/1.1 403 = GSSException: No valid credentials provided (Mechanism level: Failed to = find any Kerberos credentails)

Cache-Control: = must-revalidate,no-cache,no-store

Date: Thu, 22 = Oct 2015 07:38:52 GMT

Pragma: = no-cache

Date: Thu, 22 = Oct 2015 07:38:52 GMT

Pragma: = no-cache

Content-Type: = text/html; charset=3Diso-8859-1

Set-Cookie: = hadoop.auth=3D; Path=3D/; Expires=3DThu, 01-Jan-1970 00:00:00 GMT; = HttpOnly

Content-Length: = 1561

Server: = Jetty(6.1.26.hwx)

 

<html>

<head>

<meta = http-equiv=3D"Content-Type" content=3D"text/html; = charset=3DISO-8859-1"/>

<title>Error 403 GSSException: No valid = credentials provided (Mechanism level: Failed to find any Kerberos = credentails)</title>

</head>

<body><= h2>HTTP ERROR 403</h2>

<p>Problem= accessing /webhdfs/v1. Reason:

<pre> = ;   GSSException: No valid credentials provided (Mechanism = level: Failed to find any Kerberos = credentails)</pre></p><hr = /><i><small>Powered by = Jetty://</small></i><br/>

 

</body>

</html>

 

As for the = command line shown in the url . Here is what I am trying on the KDC = machine.

kinit = admin/admin

After entering = the password, it exited without problems

curl -i = --negotiate -u admin http://sandbox.hortonworks.com:50070/webhdfs/v1?op=3Dge= thomedirectory and enter the = password for the subsequent prompt.

It also returns = ok. I can see the response’s content is = {“Path”:”/user/admin”}

However if I = change the command line to be curl -i --negotiate -u admin http://xxx.cloudapp.net:50070/webhdfs/v1?op=3Dgethomedi= rectory, it failed with

 

<html>

<head>

<meta = http-equiv=3D"Content-Type" content=3D"text/html; = charset=3DISO-8859-1"/>

<title>Err= or 401 Authentication required</title>

</head>

<body><= h2>HTTP ERROR 401</h2>

<p>Problem= accessing /webhdfs/v1. Reason:

<pre> = ;   Authentication = required</pre></p><hr = /><i><small>Powered by = Jetty://</small></i><br/>

</body>

</html>

 

Any = ideas?

 

Thanks,

Jingfei

From: Anu Engineer [mailto:aengineer@hortonworks.com]
Sent: = Thursday, October 22, 2015 12:16 PM
To: user@hadoop.apache.org
Subject: Re: = GSSException: No valid credentials provided (Mechanism level: Failed to = find any Kerberos credentails)

 

D= id you try out with curl and it did not work at all ?.  It is = slightly easier to debug with command line than a browser like = IE.

&= nbsp;

<= a = href=3D"https://hadoop.apache.org/docs/r2.6.0/hadoop-auth/Examples.html" = target=3D"_blank">https://hadoop.apache.org/docs/r2.6.0/hadoop-auth/Examp= les.html

&= nbsp;

P= lease try getting a ticket and try to connect the cluster using it and = if does not work, could you please post your traces. 

T= his will allow us rule out the issue that you might not have a valid = kerberos ticket in the first place.

&= nbsp;

&= nbsp;

T= hanks

A= nu

&= nbsp;

&= nbsp;

From: = Jingfei = Hu
Reply-To: "user@hadoop.apache.org"
Date: = Wednesday, October 21, 2015 at 7:30 PM
To: "user@hadoop.apache.org"
Subject: = RE: GSSException: No valid credentials provided (Mechanism level: = Failed to find any Kerberos credentails)

&= nbsp;

The links Anu = provides seems to be no much helpful for my problem. I would give more = context here.

 

I am using HDP = on Azure. Its version is 2.3. After several attempts, I’m finally = able to enable Kerberos for my HDP deployment. Now I am trying to use = webhdfs to access hdfs files in IE.

http://xxx.cloudapp.net:50070/webhdfs/v1?op=3Dgethomedi= rectory.

Then IE pops up = a windows to ask for user name and password. Then I enter the admin = principal in KDC domain style (admin/admin@DOMAIN.com) along with the correct = password. However it gives me
GSSException: No valid credentials = provided (Mechanism level: Failed to find any Kerberos = credentials)

Before enabling = Kerberos, I can access this without entering anything because its = security is off.

Now how am i = gonna do to make it work with Kerberos enabled?

 

Jingfei

 

From: Anu Engineer = [mailto:aengineer@hortonworks.com]
Sent: = Wednesday, October 21, 2015 11:42 PM
To: user@hadoop.apache.org
Subject: Re: = GSSException: No valid credentials provided (Mechanism level: Failed to = find any Kerberos credentails)

 

T= his usually means that you don’t have valid Kerberos = credentials.

&= nbsp;

P= lease look at the authentication section at this URL https://hadoop.apache.org/docs/r1.0.4/webhdfs.html&= nbsp;or here is another example. 

<= a = href=3D"http://www.cloudera.com/content/www/en-us/documentation/archive/c= dh/4-x/4-3-2/CDH4-Security-Guide/cdh4sg_topic_7_2.html" = target=3D"_blank">http://www.cloudera.com/content/www/en-us/documentation= /archive/cdh/4-x/4-3-2/CDH4-Security-Guide/cdh4sg_topic_7_2.html

&= nbsp;

T= hanks

A= nu

&= nbsp;

&= nbsp;

From: = Jingfei = Hu
Reply-To: "user@hadoop.apache.org"
Date: = Tuesday, October 20, 2015 at 10:45 PM
To: "user@hadoop.apache.org"
Subject: = GSSException: No valid credentials provided (Mechanism level: Failed = to find any Kerberos credentails)

&= nbsp;

Hi = team,

Is there anyone = who knows what this exception means? What should I do to fix this?  = I am trying to access

 

http://xxx.cloudapp.net:50070/webhdfs/v1/?op=3Dliststat= us

 

I just enabled = the Kerberos for the HDFS deployment.

 

Thanks,

Jingfei

 

------=_NextPart_000_0207_01D1109C.F1CA3AC0--