hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7723) Quota By Storage Type namenode implemenation
Date Tue, 10 Feb 2015 01:58:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313368#comment-14313368
] 

Arpit Agarwal commented on HDFS-7723:
-------------------------------------

This patch is still too large to review comfortably! In the future it would be good to do
this scope of work incrementally in smaller changes.

Initial feedback:
# More of a question: In many places where setQuota takes the (nsQuota, dsQuota, type) triplet
as parameter, nsQuota should be QUOTA_DONT_SET if type is non-null. e.g. in unprotectedSetQuota
you throw an exception if types is non-NULL and nsQuota is non-zero. However methods where
you take (nsQuota, dsQuota and EnumSet) can have any combination. Is this correct?
# Error message in FsImage#updateCountForQuota is wrong. A quota violation in the image is
not necessarily a bug. The existing "BUG" error messages are also wrong. I filed HDFS-7757
to fix them. Also this loop may be a performance issue when loading millions of INodes. Since
all it does is check violation and log a message, we should just remove it.
# FSImageFormat.java:915 - did you miss passing typeCounts to copyWithQuota constructor?
# QuotaCounts: typeSpaces and typeCounts are used interchangeably. We should probably name
them consistently.
# Nitpick: Rename anyGreatOrEqual to anyGreaterOrEqual
# You renamed DEFAULT_DISKSPACE_QUOTA to DEFAULT_SPACE_QUOTA. DEFAULT_STORAGE_SPACE_QUOTA
would be easier to understand. Comments like _@return the namespace and diskspace consumed_
can also be edited to use storage space instead of diskspace. We can probably do this in a
separate patch too, the current patch is large enough already!

I'll take a look at the rest of the files tomorrow.


> Quota By Storage Type namenode implemenation
> --------------------------------------------
>
>                 Key: HDFS-7723
>                 URL: https://issues.apache.org/jira/browse/HDFS-7723
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7723.0.patch, HDFS-7723.1.patch, HDFS-7723.2.patch, HDFS-7723.3.patch,
HDFS-7723.4.patch
>
>
> This includes: 1) new editlog to persist quota by storage type op 2) corresponding fsimage
load/save the new op. 3) QuotaCount refactor to update usage of the storage types for quota
enforcement 4) Snapshot support 5) Unit test update



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message