hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang Xu (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-893) Synchronization other FSNamesystem information to BackupNode
Date Tue, 12 Jan 2010 15:17:54 GMT
Synchronization other FSNamesystem information to BackupNode

                 Key: HDFS-893
                 URL: https://issues.apache.org/jira/browse/HDFS-893
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: name-node
            Reporter: Wang Xu

Backup Node (BN) in trunk brings "warm" standby (HADOOP-4539), it synchronizes namespace information
from Main NameNode, and leaves DataNode information, Block Operation (such as replication),
and lease information unsynchronized. 

We (team from China Mobile Research Instatitute) had also some experimental on Namesystem
schronization ( some information has been post at: http://gnawux.info/hadoop/2010/01/pratice-of-namenode-cluster-for-hdfs-ha/
). Currently, we are thinking about migrate our code to re-based on Backup Node, and we think
it might convenience the maintenance of code and contributing back to community.

As NN in current trunk can log all the namespace changes as journals to BN, the rest information
needed to be synchronized includes:

 - Changes in DataNode Map: information collected from DataNode heartbeats and reports. Together
with namespace info, the blocks map can also be calculated.
 - Lease information: leases are updated by clients periodically or when it lock/unlock file
for write.
 - Block operation's status, such as pending replication block list: these information are
depends on random number, thus it may lead to inconsistency if do not synchronize them.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message