hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7584) Enable Quota Support for Storage Types
Date Tue, 27 Jan 2015 02:50:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292891#comment-14292891
] 

Arpit Agarwal edited comment on HDFS-7584 at 1/27/15 2:50 AM:
--------------------------------------------------------------

[~xyao], considering the size of the patch do you think we should split it into at least two
smaller patches. I can think of at least one natural split:
# Part 1 is the the API,  protocol and tool changes.
# Part 2 is the NameNode implementation.

In any case I started looking at the NN changes. Some initial feedback below, mostly nitpicks.
I will look at the rest later this week:

# I did not understand the todo in INode.java:464. If it is something that would be broken
and is not too hard to fix perhaps we should include it in the same checkin? This is perhaps
another argument for splitting into two patches at a higher level.
# QuotaCounts: It has four telescoping constructors, all private. It is a little confusing.
Can we simplify the constructors? e.g. the default constructor can be replaced with initializers.
# QuotaCounts: {{typeSpaces}} and {{typeCounts}} are used interchangeably. We should probably
name them consistently.
# NameNodeLayoutVersion: description of the new layout is too terse, probably unintentional.
# Could you please add a short comment for {{ONE_NAMESPACE}}? I realize it was even more confusing
before, thanks for adding the static initializer.
# {{INode.getQuotaCounts}} - don't need local variable {{qc}}.
# Nitpick, optional: {{EnumCounters.allLessOrEqual}} and {{.anyGreatOrEqual}} - can we use
foreach loop?
# DFSAdmin.java: _The space quota is set onto storage type_ should be _The storage type specific
quota is set when ..._
# Unintentional whitespace changes in Quota.java?



was (Author: arpitagarwal):
[~xyao], considering the size of the patch do you think we should split it into at least two
smaller patches. I can think of at least one natural split:
# Part 1 is the the API,  protocol and tool changes.
# Part 2 is the NameNode implementation.

In any case I started looking at the NN changes. Some initial feedback below, I will look
at the rest later this week:

# I did not understand the todo in INode.java:464. If it is something that would be broken
and is not too hard to fix perhaps we should include it in the same checkin? This is perhaps
another argument for splitting into two patches at a higher level.
# QuotaCounts: It has four telescoping constructors, all private. It is a little confusing.
e.g. the default constructor can be replaced with initializers.
# QuotaCounts: {{typeSpaces}} and {{typeCounts}} are used interchangeably. We should probably
name them consistently.
# NameNodeLayoutVersion: description of the new layout is too terse, probably unintentional.
# Could you please add a short comment for {{ONE_NAMESPACE}}? I realize it was even more confusing
before, thanks for adding the static initializer.
# {{INode.getQuotaCounts}} - don't need local variable {{qc}}.
# Nitpick, optional: {{EnumCounters.allLessOrEqual}} and {{.anyGreatOrEqual}} - can we use
foreach loop?
# DFSAdmin.java: _The space quota is set onto storage type_ should be _The storage type specific
quota is set when ..._
# Unintentional whitespace changes in Quota.java?


> Enable Quota Support for Storage Types
> --------------------------------------
>
>                 Key: HDFS-7584
>                 URL: https://issues.apache.org/jira/browse/HDFS-7584
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, namenode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7584 Quota by Storage Type - 01202015.pdf, HDFS-7584.0.patch,
HDFS-7584.1.patch, HDFS-7584.2.patch, HDFS-7584.3.patch, HDFS-7584.4.patch, editsStored
>
>
> Phase II of the Heterogeneous storage features have completed by HDFS-6584. This JIRA
is opened to enable Quota support of different storage types in terms of storage space usage.
This is more important for certain storage types such as SSD as it is precious and more performant.

> As described in the design doc of HDFS-5682, we plan to add new quotaByStorageType command
and new name node RPC protocol for it. The quota by storage type feature is applied to HDFS
directory level similar to traditional HDFS space quota. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message