hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4988) Datanode must support all the volumes as individual storages
Date Thu, 26 Sep 2013 20:36:06 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arpit Agarwal updated HDFS-4988:
--------------------------------

    Attachment: HDFS-4988.09.patch

{quote}
So, for the newly added datanodeID/storageID, I suggest to use datanodeUuid/storageUuid instead
of renaming them later.
{quote}
I don't feel strongly about it either. Whichever one makes reviewing easy is fine by me. Although
{{FsVolumeSpi#getStorageID}} method looks redundant. {{getStorageID}} is only called on known
{{FsVolumeImpl}} objects so I just removed it. The other place is in StorageDirectory where
I updated it to StorageUuid.

{quote}
Could you add TODO comments in the code if we don't have it yet?
{quote}
Done, will file Jiras soon.

{quote}
the following new added code in setFieldsFromProperties() in DataStorage.java are actually
removing verification of storageID (now is DatanodeUUID) that from VERSION file. Do we think
this is unnecessary now?
{quote}
Junping, sorry for missing your comment and thanks for catching the bug! Can you verify the
updated patch addresses your comment?
                
> Datanode must support all the volumes as individual storages
> ------------------------------------------------------------
>
>                 Key: HDFS-4988
>                 URL: https://issues.apache.org/jira/browse/HDFS-4988
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Suresh Srinivas
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-4988.01.patch, HDFS-4988.02.patch, HDFS-4988.05.patch, HDFS-4988.06.patch,
HDFS-4988.07.patch, HDFS-4988.08.patch, HDFS-4988.09.patch
>
>
> Currently all the volumes on datanode is reported as a single storage. This change proposes
reporting them as individual storage. This requires:
> # A unique storage ID for each storage
> #* This needs to be generated during formatting
> # There should be an option to allow existing disks to be reported as single storage
unit for backward compatibility.
> # A functionality is also needed to split the existing all volumes as single storage
unit to to individual storage units.
> # -Configuration must allow for each storage unit a storage type attribute. (Now HDFS-5000)-
> # Block reports must be sent on a per storage basis. In some cases (such memory tier)
block reports may need to be sent more frequently. That means block reporting period must
be on a per storage type basis.
> My proposal is for new clusters to configure volumes by default as separate storage unit.
Lets discuss.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message