hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2559) Clean up Storage/StorageInfo classes after federation
Date Thu, 17 Nov 2011 02:53:51 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-2559:
------------------------------

    Attachment: hdfs-2559.txt

The patch looks large but is actually conceptually simple:
- StorageInfo now just has cluster-wide information (ie clusterId and layoutVersion)
- NamespaceInfo extends from StorageInfo and adds the info about a particular namespace (ie
namespaceId and cTime)
- StorageSliceInfo extends from NamespaceInfo and adds the info about the DN-side storage
slice (ie blockpool id)

The rest of the changes are basically changing the usages of StorageInfo to more specifically
refer to the right storage info type.

This change gets Federation closer to the design doc where a NN may actually reference multiple
block pools, etc. The other goal of this JIRA is to make the initialization/lifecycle of the
DataNode a little cleaner in the future.
                
> Clean up Storage/StorageInfo classes after federation
> -----------------------------------------------------
>
>                 Key: HDFS-2559
>                 URL: https://issues.apache.org/jira/browse/HDFS-2559
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>    Affects Versions: 0.23.1
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-2559.txt
>
>
> After federation, there are many different identifiers floating around in a cluster:
cluster ID, namespace ID, block pool ID, plus the versioning identifiers cTime and layoutVersion.
Some of these are cluster-wide, whereas others may be different within a DN as it talks to
different NNs (see the federation design doc for more info). However, the current class heirarchy
in trunk has DataStorage (the DN-wide storage class) inherit from Storage which inherits from
StorageInfo, which contains a namespaceID. Thus, DataStorage has this unused member which
confuses things.
> This JIRA is to split up StorageInfo into separate classes/interfaces so that only the
necessary identifiers are used by each piece of the DN.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message