hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Sirianni (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-5454) DataNode UUID should be assigned prior to FsDataset initialization
Date Mon, 04 Nov 2013 17:09:18 GMT
Eric Sirianni created HDFS-5454:

             Summary: DataNode UUID should be assigned prior to FsDataset initialization
                 Key: HDFS-5454
                 URL: https://issues.apache.org/jira/browse/HDFS-5454
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode
    Affects Versions: 2.2.0
            Reporter: Eric Sirianni
            Assignee: Arpit Agarwal
            Priority: Minor

The DataNode's UUID ({{DataStorage.getDatanodeUuid()}} field) is NULL at the point where the
{{FsDataset}} object is created ({{DataNode.initStorage()}}.  

As the {{DataStorage}} object is an input to the {{FsDataset}} factory method, it is desirable
for it to be fully populated with a UUID at this point.  In particular, our {{FsDatasetSpi}}
implementation relies upon the DataNode UUID as a key to access our underlying block storage

This also appears to be a regression compared to Hadoop 1.x - our 1.x {{FSDatasetInterface}}
plugin has a non-NULL UUID on startup.  I haven't fully traced through the code, but I suspect
this came from the {{BPOfferService}}/{{BPServiceActor}} refactoring to support federated

With HDFS-5448, the DataNode is now responsible for generating its own UUID.  This greatly
simplifies the fix.  Move the UUID check/generation in from {{DataNode.createBPRegistration()}}
to {{DataNode.initStorage()}}.  This more naturally co-locates UUID generation immediately
subsequent to the read of the UUID from the {{DataStorage}} properties file.

  private void initStorage(final NamespaceInfo nsInfo) throws IOException {
    // ...

      final String bpid = nsInfo.getBlockPoolID();
      //read storage info, lock data dirs and transition fs state if necessary
      storage.recoverTransitionRead(this, bpid, nsInfo, dataDirs, startOpt);

    // ...

This message was sent by Atlassian JIRA

View raw message