hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3988) The elephant should remember names, not numbers.
Date Thu, 21 Aug 2008 14:10:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624378#action_12624378
] 

Allen Wittenauer commented on HADOOP-3988:
------------------------------------------

We recently tried a "new kind" of fail over in our environment.  Rather than having a static
IP for the name node, we attempted to use DNS CNAMEs to move the name node from one node to
another.  We discovered that the data nodes continually attempted to contact the old machine
even though DNS pointed to the new machine.

Since we configure a host name in hadoop, I would expect that the data nodes would at some
point drop their cache of the IP and re-resolve.  However, this never happened.

I'd like to see either an option or just the default to be when a name is given in a configuration
file, Hadoop always does a host name resolution on that entry prior to connection.  The operating
system should be able to handle the job of caching any addresses that need to be cached, either
through a mechanism like nscd or through a fully-blooded, local DNS cache.

> The elephant should remember names, not numbers.
> ------------------------------------------------
>
>                 Key: HADOOP-3988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3988
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.17.2
>            Reporter: Allen Wittenauer
>
> The name node and the data node should not cache the resolution of host names, as doing
so prevents the use of DNS CNAMEs for any sort of fail over capability.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message