hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8292) TableMapping does not refresh when topology is updated
Date Wed, 09 May 2012 16:15:49 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alejandro Abdelnur updated HADOOP-8292:
---------------------------------------

    Attachment: HADOOP-8292.patch

new patch addressing Todd's comments.

the new patch removes the 'hot path' by using a daemon thread that does the check/reload and
swapping the Map using an AtomicRef. It does a 2 seconds lag check as you suggested. Regarding
the clearCache(), the underlaying impl is a ConcurrentMap, so that would not be an issue (still,
now the clearcache is called after swapping the new Map in the daemon thread).

Regarding your last question on how it may impact HDFS, I don't know I'll dig.
                
> TableMapping does not refresh when topology is updated
> ------------------------------------------------------
>
>                 Key: HADOOP-8292
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8292
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Philip Zeyliger
>            Assignee: Alejandro Abdelnur
>         Attachments: HADOOP-8292.patch, HADOOP-8292.patch
>
>
> HADOOP-7030 introduced TableMapping, an implementation of DNSToSwitchMapping which uses
a file to map from IPs/hosts to their racks.  It's intended to replace ScriptBasedMapping
for cases where the latter was just a complicated way of looking up the rack in a file.
> Though there was discussion of it on the JIRA, the TableMapping implementation is not
'refreshable'.  i.e., if you want to add a host to your cluster, and that host wasn't in the
topology file to begin with, it will never be added.
> TableMapping should refresh, either based on a command that can be executed, or, perhaps,
if the file on disk changes.
> I'll also point out that TableMapping extends CachedDNSToSwitchMapping, but, since it
does no refreshing, I don't see what the caching gets you: I think the cache ends up being
a second copy of the underlying map, always.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message