hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
Date Thu, 30 Jan 2014 15:48:10 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886680#comment-13886680

Tsz Wo (Nicholas), SZE commented on HDFS-5754:

Thanks a lot for update.  Patch looks good.  Some minor comments:

- When a StorageInfo object is created, the storage type is known and remains unchanged. 
Thus, StorageInfo.storageType should be final.

- Let's add NodeType as a parameter to PBHelper.convert(StorageInfoProto info) since the caller
knows the NodeType.

> Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion 
> ------------------------------------------------------------------------
>                 Key: HDFS-5754
>                 URL: https://issues.apache.org/jira/browse/HDFS-5754
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Brandon Li
>         Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch,
HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch,
HDFS-5754.010.patch, HDFS-5754.012.patch
> Currently, LayoutVersion defines the on-disk data format and supported features of the
entire cluster including NN and DNs.  LayoutVersion is persisted in both NN and DNs.  When
a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion.
 Also, a DN with a different LayoutVersion than NN cannot register with the NN.
> We propose to split LayoutVersion into two independent values that are local to the nodes:
> - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format
of FSImage, editlog and the directory structure.
> - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format
of block data file, metadata file, block pool layout, and the directory structure.  
> The LayoutVersion check will be removed in DN registration.  If NamenodeLayoutVersion
or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported
and downgrade is not.

This message was sent by Atlassian JIRA

View raw message