hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Sirianni (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5448) Datanode should generate its ID on first registration
Date Mon, 04 Nov 2013 16:43:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812989#comment-13812989
] 

Eric Sirianni commented on HDFS-5448:
-------------------------------------

Actually, rather than having a potentially temporary UUID in the {{DataStorage}} object, (as
I suggested above), a better solution would be to initialize the UUID in {{DataNode.initStorage()}}:

{code}
  private void initStorage(final NamespaceInfo nsInfo) throws IOException {
    // ...

      final String bpid = nsInfo.getBlockPoolID();
      //read storage info, lock data dirs and transition fs state if necessary
      storage.recoverTransitionRead(this, bpid, nsInfo, dataDirs, startOpt);
      
      // SUGGESTED NEW PLACE TO CHECK DATANODE UUID
      checkDatanodeUuid();

    // ...
  }
{code}

Arpit - is there any issue with moving the call to {{checkDatanodeUuid()}} to {{initStorage()}}
as suggested above?

> Datanode should generate its ID on first registration
> -----------------------------------------------------
>
>                 Key: HDFS-5448
>                 URL: https://issues.apache.org/jira/browse/HDFS-5448
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: Heterogeneous Storage (HDFS-2832)
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>             Fix For: Heterogeneous Storage (HDFS-2832)
>
>         Attachments: h5448.01.patch, h5448.03.patch, h5448.04.addendum.patch, h5448.04.patch
>
>
> Prior to the heterogeneous storage feature, each Datanode had a single storage ID which
was generated by the Namenode on first registration. The storage ID used fixed Datanode identifiers
like IP address and port, so that in a federated cluster, for example, all NameNodes would
generate the same storage ID.
> With Heterogeneous storage, we have replaced the storage ID with a per-datanode identifier
called the Datanode-UUID. The Datanode UUID is also assigned by a NameNode on first registration.
In a federated cluster with multiple namenodes, there are two ways to ensure a unique Datanode
UUID allocation:
> # Synchronize initial registration requests from the BPServiceActors. If a Datanode UUID
is already assigned we don't need to synchronize.
> # The datanode assigns itself a UUID on initialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message