hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7723) Quota By Storage Type namenode implemenation
Date Tue, 10 Feb 2015 01:58:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313368#comment-14313368

Arpit Agarwal commented on HDFS-7723:

This patch is still too large to review comfortably! In the future it would be good to do
this scope of work incrementally in smaller changes.

Initial feedback:
# More of a question: In many places where setQuota takes the (nsQuota, dsQuota, type) triplet
as parameter, nsQuota should be QUOTA_DONT_SET if type is non-null. e.g. in unprotectedSetQuota
you throw an exception if types is non-NULL and nsQuota is non-zero. However methods where
you take (nsQuota, dsQuota and EnumSet) can have any combination. Is this correct?
# Error message in FsImage#updateCountForQuota is wrong. A quota violation in the image is
not necessarily a bug. The existing "BUG" error messages are also wrong. I filed HDFS-7757
to fix them. Also this loop may be a performance issue when loading millions of INodes. Since
all it does is check violation and log a message, we should just remove it.
# FSImageFormat.java:915 - did you miss passing typeCounts to copyWithQuota constructor?
# QuotaCounts: typeSpaces and typeCounts are used interchangeably. We should probably name
them consistently.
# Nitpick: Rename anyGreatOrEqual to anyGreaterOrEqual
would be easier to understand. Comments like _@return the namespace and diskspace consumed_
can also be edited to use storage space instead of diskspace. We can probably do this in a
separate patch too, the current patch is large enough already!

I'll take a look at the rest of the files tomorrow.

> Quota By Storage Type namenode implemenation
> --------------------------------------------
>                 Key: HDFS-7723
>                 URL: https://issues.apache.org/jira/browse/HDFS-7723
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7723.0.patch, HDFS-7723.1.patch, HDFS-7723.2.patch, HDFS-7723.3.patch,
> This includes: 1) new editlog to persist quota by storage type op 2) corresponding fsimage
load/save the new op. 3) QuotaCount refactor to update usage of the storage types for quota
enforcement 4) Snapshot support 5) Unit test update

This message was sent by Atlassian JIRA

View raw message