Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 85CC01040F for ; Thu, 25 Apr 2013 19:35:30 +0000 (UTC) Received: (qmail 18673 invoked by uid 500); 25 Apr 2013 19:35:23 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 18495 invoked by uid 500); 25 Apr 2013 19:35:23 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 18488 invoked by uid 99); 25 Apr 2013 19:35:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 19:35:23 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: 216.145.54.171 is neither permitted nor denied by domain of daryn@yahoo-inc.com) Received: from [216.145.54.171] (HELO mrout1.yahoo.com) (216.145.54.171) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 19:35:18 +0000 Received: from GQ1-EX10-CAHT14.y.corp.yahoo.com (gq1-ex10-caht14.corp.gq1.yahoo.com [10.73.119.195]) by mrout1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id r3PJYTMs044298 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Thu, 25 Apr 2013 12:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1366918470; bh=UcQhQ4MlXxYQwzYNHF0Wd+uM0wirVNLAU07XSKBsueA=; h=From:To:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=ujWXnxbt3jqstVo4rYNFfI8pXF0EZghkbZA7EDIbmIv6avXqNerLaWncEXPLOTCP1 iDYt+eyEE41uLLnh9Z3RjuvU+mNor5lr/a38+A410Q/C7akwU9fiCntgDj0YsnUPnu IkKi2OYxU4tWjxHDWUVlpNRWbZy83QD3LP7LnrXE= Received: from GQ1-MB01-02.y.corp.yahoo.com ([fe80::a049:b5af:9055:ada6]) by GQ1-EX10-CAHT14.y.corp.yahoo.com ([fe80::191f:3466:d1b7:3b3d%14]) with mapi id 14.03.0123.003; Thu, 25 Apr 2013 12:34:29 -0700 From: Daryn Sharp To: "" Subject: Re: How to connect to hadoop through ssh tunnel and kerberos authentication Thread-Topic: How to connect to hadoop through ssh tunnel and kerberos authentication Thread-Index: AQHOQXcucqk+tWXDlkWtP2MSH1MR1JjnyjaA Date: Thu, 25 Apr 2013 19:34:28 +0000 Message-ID: <3AFF04F7-584B-4F25-A0A9-E6ECB9EDF895@yahoo-inc.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.74.226.61] Content-Type: multipart/alternative; boundary="_000_3AFF04F7584B4F25A0A9E6ECB9EDF895yahooinccom_" MIME-Version: 1.0 X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 918470000 X-Virus-Checked: Checked by ClamAV on apache.org --_000_3AFF04F7584B4F25A0A9E6ECB9EDF895yahooinccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The important part of the error is "Cannot get kdc for realm CORP.EBAY.COM<= http://CORP.EBAY.COM>". Check if the gateway's /etc/krb5.conf has an entry= for CORP.EBAY.COM in the [realms] section. Or if yo= u actually have appropriate dns service records for kerberos, you can use "= dns_lookup_kdc =3D true". Daryn On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote: Hi all, I could connect to hadoop cluster by ssh tunnel before when there's no kerb= eros authentication. Now our cluster need to upgrade to kerberos authentica= tion. I try to connect to it by ssh tunnel again. But failed. Could anyone guide me to do that ? Is there any tutorial for this ? Here's what I did. 1. create a forwardable ticket in my client machine. 2. edit ~/.ssh/config file GSSAPIAuthentication yes GSSAPIDelegateCredentials yes 3. execute command "ssh -N -D 3600 gateway_host " to create a ssh connec= tion to my gateway host 4. config my core-site.xml file for ssh tunnel connection hadoophack.tunnel.port 3600 If users connect through a SOCKS proxy, we don't want their SocketFactory settings interfering with the socket factory associated with the actual daemons. hadoop.rpc.socket.factory.class.default org.apache.hadoop.net.SocksSocketFactory true And there's the error message when I run "hadoop fs -ls /" 13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActionExc= eption as:jianfezhang@CORP.EBAY.COM = cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSS= Exception: No valid credentials provided (Mechanism level: Cannot get kdc f= or realm CORP.EBAY.COM)] 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for= jianfezhang@CORP.EBAY.COM 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login f= or jianfezhang@CORP.EBAY.COM 13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActionExc= eption as:jianfezhang@CORP.EBAY.COM = cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSS= Exception: No valid credentials provided (Mechanism level: Cannot get kdc f= or realm CORP.EBAY.COM)] 13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to re-= login since the last re-login was attempted less than 600 seconds before. 13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActionExc= eption as:jianfezhang@CORP.EBAY.COM = cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSS= Exception: No valid credentials provided (Mechanism level: Cannot get kdc f= or realm CORP.EBAY.COM)] -- Best Regards Jeff Zhang --_000_3AFF04F7584B4F25A0A9E6ECB9EDF895yahooinccom_ Content-Type: text/html; charset="us-ascii" Content-ID: <644DB805FE32C445AB3161CA50513BAE@yforest.corp.yahoo.com> Content-Transfer-Encoding: quoted-printable
The important part of the error is "Cannot get kdc for realm = ;CORP.EBAY.COM".  Check if t= he gateway's /etc/krb5.conf has an entry for CORP.EBAY.COM in the [realms] section.=  Or if you actually have appropriate dns service records for kerberos= , you can use "dns_lookup_kdc =3D true".

Daryn

On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:

Hi all,


I could connect to hadoop cluster by ssh tunnel before when there's no kerb= eros authentication. Now our cluster need to upgrade to kerberos authentica= tion. I try to connect to it by ssh tunnel again. But failed.

Could anyone guide me to do that ? Is there any tutorial for this ?

Here's what I did.

  1. create a forwardable ticket in my client machine.

  2. edit ~/.ssh/config file

    GSSAPIAuthentication yes

    GSSAPIDelegateCredentials yes

  3. execute command "ssh -N -D 3600 gateway_host " to create a ssh co= nnection to my gateway host

  4. config my core-site.xml file for ssh tunnel connection
<property>
        <name>hadoophack.tunnel.port</name>
        <value>3600</value>
</proper=
ty>

<propert=
y>
    <des=
cription>If users connect through a SOCKS proxy, we don't
      want their SocketFactory settings interfering with the socket
      factory associated with the actual daemons.</description>
    <nam=
e>hadoop.rpc.socket.factory.class.default</name>
    <val=
ue>org.apache.hadoop.net.SocksSocketFactory</value>
    <fin=
al>true</final>
</proper=
ty>

And there's the error message when I run "hadoop fs -ls /"

13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActi= onException as:jianfezhang@CORP.EBAY.= COM cause:javax.security.sasl.SaslException: GSS initiate failed [Cause= d by GSSException: No valid credentials provided (Mechanism level: Cannot g= et kdc for realm CORP.EBAY.COM)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logou= t for jianfezhang@CORP.EBAY.COM
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-lo= gin for jianfezhang@CORP.EBAY.COM<= /div>
13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActi= onException as:jianfezhang@CORP.EBAY.= COM cause:javax.security.sasl.SaslException: GSS initiate failed [Cause= d by GSSException: No valid credentials provided (Mechanism level: Cannot g= et kdc for realm CORP.EBAY.COM)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting t= o re-login since the last re-login was attempted less than 600 seconds befo= re.
13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActi= onException as:jianfezhang@CORP.EBAY.= COM cause:javax.security.sasl.SaslException: GSS initiate failed [Cause= d by GSSException: No valid credentials provided (Mechanism level: Cannot g= et kdc for realm CORP.EBAY.COM)]

--
Best Regards

Jeff Zhang

--_000_3AFF04F7584B4F25A0A9E6ECB9EDF895yahooinccom_--