hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5454) DataNode UUID should be assigned prior to FsDataset initialization
Date Sun, 15 Dec 2013 02:17:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848512#comment-13848512
] 

Hadoop QA commented on HDFS-5454:
---------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12618793/HDFS-5454.03.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5723//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5723//console

This message is automatically generated.

> DataNode UUID should be assigned prior to FsDataset initialization
> ------------------------------------------------------------------
>
>                 Key: HDFS-5454
>                 URL: https://issues.apache.org/jira/browse/HDFS-5454
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: 3.0.0
>            Reporter: Eric Sirianni
>            Priority: Minor
>         Attachments: HDFS-5454.01.patch, HDFS-5454.02.patch, HDFS-5454.03.patch
>
>
> The DataNode's UUID ({{DataStorage.getDatanodeUuid()}} field) is NULL at the point where
the {{FsDataset}} object is created ({{DataNode.initStorage()}}.  
> As the {{DataStorage}} object is an input to the {{FsDataset}} factory method, it is
desirable for it to be fully populated with a UUID at this point.  In particular, our {{FsDatasetSpi}}
implementation relies upon the DataNode UUID as a key to access our underlying block storage
device.
> This also appears to be a regression compared to Hadoop 1.x - our 1.x {{FSDatasetInterface}}
plugin has a non-NULL UUID on startup.  I haven't fully traced through the code, but I suspect
this came from the {{BPOfferService}}/{{BPServiceActor}} refactoring to support federated
namenodes.
> With HDFS-5448, the DataNode is now responsible for generating its own UUID.  This greatly
simplifies the fix.  Move the UUID check/generation in from {{DataNode.createBPRegistration()}}
to {{DataNode.initStorage()}}.  This more naturally co-locates UUID generation immediately
subsequent to the read of the UUID from the {{DataStorage}} properties file.
> {code}
>   private void initStorage(final NamespaceInfo nsInfo) throws IOException {
>     // ...
>       final String bpid = nsInfo.getBlockPoolID();
>       //read storage info, lock data dirs and transition fs state if necessary
>       storage.recoverTransitionRead(this, bpid, nsInfo, dataDirs, startOpt);
>       
>       // SUGGESTED NEW PLACE TO CHECK DATANODE UUID
>       checkDatanodeUuid();
>     // ...
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message