hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5448) Synchronize NN registrations across BPServiceActors
Date Fri, 01 Nov 2013 21:31:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811680#comment-13811680

Arpit Agarwal commented on HDFS-5448:

No drawback, unless we want to let NameNodes manage the UUID assignment.

An issue with relying on the NameNode handshake to assign the UUID is that there is a lengthy
period during DataNode initialization where the DatanodeUuid is still NULL. In particular,
when the FsDatasetSpi is created in initStorage(), the DatanodeUuid is NULL. The DatanodeUuid
is not set afterwards until bpRegistrationSucceeded().
[~sirianni], I believe this is not very different from today where the storageID remains unassigned
until after the first handshake.

> Synchronize NN registrations across BPServiceActors
> ---------------------------------------------------
>                 Key: HDFS-5448
>                 URL: https://issues.apache.org/jira/browse/HDFS-5448
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: Heterogeneous Storage (HDFS-2832)
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: h5448.01.patch, h5448.03.patch
> Prior to the heterogeneous storage feature, each Datanode had a single storage ID which
was generated by the Namenode on first registration. The storage ID used fixed Datanode identifiers
like IP address and port, so that in a federated cluster, for example, all NameNodes would
generate the same storage ID.
> With Heterogeneous storage, we have replaced the storage ID with a per-datanode identifier
called the Datanode-UUID. The Datanode UUID is also assigned by a NameNode on first registration.
In a federated cluster with multiple namenodes, there are two ways to ensure a unique Datanode
UUID allocation:
> # Synchronize initial registration requests from the BPServiceActors. If a Datanode UUID
is already assigned we don't need to synchronize.
> # The datanode assigns itself a UUID on initialization.

This message was sent by Atlassian JIRA

View raw message