hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens
Date Tue, 24 Sep 2013 17:48:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776547#comment-13776547
] 

Colin Patrick McCabe commented on HDFS-5208:
--------------------------------------------

{code}
-      dnsToSwitchMapping.reloadCachedMappings();
+      List<String> invalidNodeNames = new ArrayList<String>(1);
+      // clear cache for nodes in IP or Hostname
+      invalidNodeNames.add(nodeReg.getIpAddr());
+      invalidNodeNames.add(nodeReg.getHostName());
+      dnsToSwitchMapping.reloadCachedMappings(invalidNodeNames);
{code}

Can we also add something like this?
{code}
+      invalidNodeNames.add(nodeReg.getPeerHostName());
{code}

It seems like the datanode could be known by any one of those three: IP address, registration
name, or hostname.

After that change, I don't see any reason why this shouldn't work.  What kind of testing have
you done?
                
> Only clear network location cache on specific nodes if invalid NetworkTopology happens
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-5208
>                 URL: https://issues.apache.org/jira/browse/HDFS-5208
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HDFS-5208-v1.patch
>
>
> After HDFS-4521, once a DN is registered with invalid networktopology, all cached rack
info in DNSToSwitchMapping will be cleared. We should only clear cache on specific nodes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message