hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7584) Enable Quota Support for Storage Types (SSD)
Date Mon, 26 Jan 2015 20:53:36 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292398#comment-14292398
] 

Arpit Agarwal commented on HDFS-7584:
-------------------------------------

Hi [~xyao], thanks for picking up this non-trivial task. Here is some initial feedback on
the patch:

# StorageType: You don't need a new constructor parameter for {{supportQuotaByStorageType}}.
You can just use {{!isTransient}}. We can add the flexibility in the future if needed.
# {{StorageType.isStorageTypeChosen}} can be removed. It looks out of place in its public
interface.
# {{StorageType.getQuotableTypes}} is oddly named. Perhaps {{typesSupportingQuota}} is better?
# You don't need the {{StorageTypeProto.NONE}} or the default value for it in {{SetQuotaRequestProto}}.
Since it is an optional field we can check for its presence with {{req.hasStorageType}} in
{{ClientNamenodeProtocolPB.setQuota}}.
# Similarly {{ClientNamenodeProtocolTranslatorPB.setQuota}} should not invoke {{setStorageType}}
if the {{type}} parameter is null.
# The loop in {{StorageType.parseStorageType(String)}} can be replaced with {{StorageType.valueOf(s.toUpperCase())}}
right?
# The loop in {{StorageType.parseStorageType(int)}} is not necessary. You can just use {{StorageType.values()[i]}}
instead. {{StorageType.values[]}} can be cached in a static context to avoid allocating it
repeatedly. The 'return null' fallback will be lost and we will bubble up an exception instead.
I checked the callers and think that will be fine.

More feedback to follow.

> Enable Quota Support for Storage Types (SSD) 
> ---------------------------------------------
>
>                 Key: HDFS-7584
>                 URL: https://issues.apache.org/jira/browse/HDFS-7584
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, namenode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7584 Quota by Storage Type - 01202015.pdf, HDFS-7584.0.patch,
HDFS-7584.1.patch, HDFS-7584.2.patch, HDFS-7584.3.patch, HDFS-7584.4.patch, editsStored
>
>
> Phase II of the Heterogeneous storage features have completed by HDFS-6584. This JIRA
is opened to enable Quota support of different storage types in terms of storage space usage.
This is more important for certain storage types such as SSD as it is precious and more performant.

> As described in the design doc of HDFS-5682, we plan to add new quotaByStorageType command
and new name node RPC protocol for it. The quota by storage type feature is applied to HDFS
directory level similar to traditional HDFS space quota. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message