hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8292) TableMapping does not refresh when topology is updated
Date Sun, 23 Sep 2012 17:29:08 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461462#comment-13461462
] 

Harsh J commented on HADOOP-8292:
---------------------------------

{code}
+  protected void clearCache() {
+    cache.clear();
+  }
{code}

Is there a way to also "invoke" a cache clearance dynamically (say via a dfsadmin command)?
Should we have one?

Patch appears alright to me. Mind checking it once again and rebasing?

Todd's question:

bq. How does this interact with the HDFS topology code which needs to check when a cluster
changes from single-rack to multi-rack? When a node's topology changes, don't we need to re-check
replication policies for all the blocks, etc? Maybe this isn't a new issue, but it's certainly
strange.

Currently the fsck begins warning as soon as the mapping changes in its perspective (when
it checks for it). Things such as balancer, etc. go crazy after that but doesn't cause a downtime
as such. Perhaps this needs to be discussed in a separate JIRA?
                
> TableMapping does not refresh when topology is updated
> ------------------------------------------------------
>
>                 Key: HADOOP-8292
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8292
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Philip Zeyliger
>            Assignee: Alejandro Abdelnur
>         Attachments: HADOOP-8292.patch, HADOOP-8292.patch
>
>
> HADOOP-7030 introduced TableMapping, an implementation of DNSToSwitchMapping which uses
a file to map from IPs/hosts to their racks.  It's intended to replace ScriptBasedMapping
for cases where the latter was just a complicated way of looking up the rack in a file.
> Though there was discussion of it on the JIRA, the TableMapping implementation is not
'refreshable'.  i.e., if you want to add a host to your cluster, and that host wasn't in the
topology file to begin with, it will never be added.
> TableMapping should refresh, either based on a command that can be executed, or, perhaps,
if the file on disk changes.
> I'll also point out that TableMapping extends CachedDNSToSwitchMapping, but, since it
does no refreshing, I don't see what the caching gets you: I think the cache ends up being
a second copy of the underlying map, always.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message