hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gurmukh Singh <gurmukh.dhil...@yahoo.com.INVALID>
Subject Re: datanode is unable to connect to namenode
Date Thu, 30 Jun 2016 08:14:13 GMT
What you are doing is correct, but datanodes, in addition to the 
dn/_HOST principal also needs nn/_HOST principal.

Follow my github for configs from workign cluster: 
https://github.com/netxillon/hadoop/tree/master/kerberos



On 30/06/16 5:54 PM, Aneela Saleem wrote:
> Thanks Vinayakumar
>
> Yes you got it right i was using different principal names i.e., 
> *nn/_HOST* for namenode and *dn/_HOST* for datanode. Setting the same 
> principal name for both datanode and namenode i.e., 
> hdfs/_HOST@platalyticsrealm solved the issue. Now datanode
> can connect to namenode successfully.
>
> So my question is, is it mandatory to have same principal name on all 
> hosts i.e., hdfs/_HOST@platalyticsrealm, because i found in many
> tutorials that the convention is to have different principals for all 
> services like
> dn/_HOST for datanode
> nn/_HOST for namenode
> sn/_HOST for secondarynamenode etc
>
> Secondly for map reduce and yarn, would that mapred-site.xml and 
> yarn-site.xml be same on all cluster nodes? just like for hdfs-site.xml
>
> Thanks
>
> On Thu, Jun 30, 2016 at 10:51 AM, Vinayakumar B 
> <vinayakumar.ba@huawei.com <mailto:vinayakumar.ba@huawei.com>> wrote:
>
>     Hi Aneela,
>
>     1. Looks like you have attached the hdfs-site.xml from
>     'hadoop-master' node. For this node datanode connection is
>     successfull as mentioned in below logs.
>
>              2016-06-29 10:01:35,700 INFO
>     SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
>     nn/hadoop-master@platalyticsrealm (auth:KERBEROS)
>
>     2016-06-29 10:01:35,744 INFO
>     SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
>     Authorization successful for nn/hadoop-master@platalyticsrealm
>     (auth:KERBEROS) for protocol=interface
>     org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol
>
>              2016-06-29 10:01:36,845 INFO
>     org.apache.hadoop.net.NetworkTopology: Adding a new node:
>     /default-rack/192.168.23.206:1004 <http://192.168.23.206:1004>
>
>     2. For the other node, 'hadoop-slave' kerberos athentication is
>     successfull, but ServiceAuthorizationManager check failed.
>
>     2016-06-29 10:01:37,474 INFO
>     SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
>     dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
>     2016-06-29 10:01:37,512 WARN
>     SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
>     Authorization failed for dn/hadoop-slave@platalyticsrealm
>     (auth:KERBEROS) for protocol=interface
>     org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol, expected
>     client Kerberos principal is nn/hadoop-slave@platalyticsrealm
>
>     2016-06-29 10:01:37,514 INFO org.apache.hadoop.ipc.Server:
>     Connection from 192.168.23.207:32807 <http://192.168.23.207:32807>
>     for protocol
>     org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol is
>     unauthorized for user dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
>     reason could be mostly, "dfs.datanode.kerberos.principal"
>     configuration in both nodes differ. I can see that this
>     configuration in hadoop-master's hdfs-site.xml set to
>     'nn/_HOST@platalyticsrealm' but it might have been set to
>     'dn/_HOST@platalyticsrealm' in hadoop-slave node's configurations.
>
>     Please change this configuration in all nodes to
>     'dn/_HOST@platalyticsrealm' and restart all NNs and DNs, and check
>     again.
>
>     If this does not help, then please share the hdfs-site.xml of
>     hadoop-slave node too.
>
>     -Vinay
>
>     *From:*Aneela Saleem [mailto:aneela@platalytics.com
>     <mailto:aneela@platalytics.com>]
>     *Sent:* 29 June 2016 21:35
>     *To:* user@hadoop.apache.org <mailto:user@hadoop.apache.org>
>     *Subject:* Fwd: datanode is unable to connect to namenode
>
>
>
>     Sent from my iPhone
>
>
>     Begin forwarded message:
>
>         *From:*Aneela Saleem <aneela@platalytics.com
>         <mailto:aneela@platalytics.com>>
>         *Date:* 29 June 2016 at 10:16:36 GMT+5
>         *To:* "sreebalineni ." <sreebalineni@gmail.com
>         <mailto:sreebalineni@gmail.com>>
>         *Subject:* *Re: datanode is unable to connect to namenode*
>
>         Attached are the log files for datanode and namenode. Also i
>         have attached hdfs-site.xml for namenode please check if there
>         are any issues in configuration file.
>
>         I have following two Kerberos Principals:
>
>         nn/hadoop-master
>
>         dn/hadoop-slave
>
>         i have copied kdc.conf and krb5.conf on both nodes. Also i
>         copied keytab file on datanode. And i have starting services
>         with principal nn/hadoop-master.
>
>         On Wed, Jun 29, 2016 at 9:35 AM, sreebalineni .
>         <sreebalineni@gmail.com <mailto:sreebalineni@gmail.com>> wrote:
>
>             Probably sharing both Name node and datanode logs may help.
>
>             On Wed, Jun 29, 2016 at 10:02 AM, Aneela Saleem
>             <aneela@platalytics.com <mailto:aneela@platalytics.com>>
>             wrote:
>
>                 Following is the result of telnet
>
>                 Trying 192.168.23.206...
>
>                 Connected to hadoop-master.
>
>                 Escape character is '^]'.
>
>                 On Wed, Jun 29, 2016 at 3:57 AM, Aneela Saleem
>                 <aneela@platalytics.com
>                 <mailto:aneela@platalytics.com>> wrote:
>
>                     Thanks Sreebalineni for the response.
>
>                     This is the result of the *netstat -a | grep 8020*
>                     command
>
>                     tcp        0  0 hadoop-master:8020      *:* LISTEN
>
>                     tcp        0  0 hadoop-master:33356
>                     hadoop-master:8020  ESTABLISHED
>
>                     tcp        0  0 hadoop-master:8020
>                      hadoop-master:33356 ESTABLISHED
>
>                     tcp        0      0 hadoop-master:55135
>                     hadoop-master:8020      TIME_WAIT
>
>                     And this is my */etc/hosts* file
>
>                     #127.0.0.1      localhost
>
>                     #127.0.1.1  vm6-VirtualBox
>
>                     192.168.23.206  hadoop-master platalytics.com
>                     <http://platalytics.com> vm6-VirtualBox
>
>                     192.168.23.207  hadoop-slave
>
>                     # The following lines are desirable for IPv6
>                     capable hosts
>
>                     ::1 ip6-localhost ip6-loopback
>
>                     fe00::0 ip6-localnet
>
>                     ff00::0 ip6-mcastprefix
>
>                     ff02::1 ip6-allnodes
>
>                     ff02::2 ip6-allrouters
>
>
>
>                     Can you please tell me what's wrong with above
>                     configuration and how can i check whether it is
>                     firewall issue?
>
>                     Thanks
>
>                     On Wed, Jun 29, 2016 at 12:11 AM, sreebalineni .
>                     <sreebalineni@gmail.com
>                     <mailto:sreebalineni@gmail.com>> wrote:
>
>                         Are you able to telnet ping. Check the
>                         firewalls as well
>
>                         On Jun 29, 2016 12:39 AM, "Aneela Saleem"
>                         <aneela@platalytics.com
>                         <mailto:aneela@platalytics.com>> wrote:
>
>                             Hi all,
>
>                             I have setup two nodes cluster with
>                             security enabled. I have everything
>                             running successful like namenode,
>                             datanode, resourcemanager, nodemanager,
>                             jobhistoryserver etc. But datanode is
>                             unable to connect to namenode, as i can
>                             see only one node on the web UI. checking
>                             logs of datanode gives following warning:
>
>                             *WARN
>                             org.apache.hadoop.hdfs.server.datanode.DataNode:
>                             Problem connecting to server:
>                             hadoop-master/192.168.23.206:8020
>                             <http://192.168.23.206:8020>*
>
>                             Rest of the things look fine. Please help
>                             me in this regard, what could be the issue?
>
>

-- 
--
Thanks and Regards

Gurmukh Singh


Mime
View raw message