hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jiang licht <licht_ji...@yahoo.com>
Subject Data node cannot talk to name node Re: Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?
Date Fri, 05 Mar 2010 03:32:23 GMT
Thanks Edward. Since the string of "reverse mapping ..." is just a warning, I guess it won't
be a issue.

Now, the namenode "A" is listening on port "a". No data node sitting on a different box can
talk to A@a to join the cluster. But assign "A" also as a datanode is ok and this datanode
can join the cluster since it sits on the same box as namenode. Telnet from other machines
to A@a fails. To verify that the slave box is allowed, I also added both IP and name (according
to boolean inHostsList(DatanodeID node, String ipAddr) in "FSNamesystem.java" which does access
control) to "slaves" and the file specified for "dfs.hosts" but I didn't use a exclude list.
Namenode box can ssh to itself and to slaves. Also I verified the IP address of slave is the
ip address listed in "slaves" and the file specified for "dfs.hosts". And the ports are open
for access from slaves on the namenode machine. For example, from slaves, telnet to 50070/50030@namenode
is allowed and by firing a "GET /" returns correct html pages. So, ssh is ok and access control
is ok. Really
 confusing. What may cause this problem which prevents data nodes from talking to namenode? 
Any thoughts?



--- On Thu, 3/4/10, Edward Capriolo <edlinuxguru@gmail.com> wrote:

From: Edward Capriolo <edlinuxguru@gmail.com>
Subject: Re: Will interactive password authentication fail talk between  namenode-datanode/jobtracker-tasktracker?
To: common-user@hadoop.apache.org
Date: Thursday, March 4, 2010, 3:23 PM

On Thu, Mar 4, 2010 at 2:01 PM, Edson Ramiro <erlfilho@gmail.com> wrote:
> You don't need DNS.
> You can use the /etc/hosts.
> I'm using it here and it's working well.
> Edson Ramiro
> On 4 March 2010 14:39, Allen Wittenauer <awittenauer@linkedin.com> wrote:
>> On 3/3/10 3:38 PM, "jiang licht" <licht_jiang@yahoo.com> wrote:
>> > Here's my question, I have to type my password (not PASSPHRASE for key)
>> due to
>> > some reverse name resolution problem when I do either SSH MASTER from
>> SLAVE or
>> > SSH SLAVE from MASTER. Since my system admin told me all ports are open
>> > between them, I am wondering will this interactive authentication
>> prevents
>> > datanode/tasktracker from talking to namenode/jobtracker?
>> If reverse name resolution isn't working, then you are going to have all
>> sorts of problems.  DNS needs to be properly configured.  [I wish I knew
>> where the whole "you don't need reverses configured for dns" thing came
>> from
>> amongst admins.]

Adding an entry to DNS adds the entry to your forward and reverse
lookup table, thus fixing/hiding the problem. Host file is not a
winning strategy long term for obvious reasons :)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message