hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8092) dfs -count -q should not consider snapshots under REM_QUOTA
Date Tue, 14 Apr 2015 11:41:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493997#comment-14493997
] 

Rakesh R commented on HDFS-8092:
--------------------------------

Thanks [~archanat] for reporting this. 

Thanks [~aw] for the interest and the comments.
bq. Snapshots should most definitely be considered part of the quota calculation. They are
not free and do take up space.
My observation is, snapshots are not considered while verifying the quota. But the {{ContentSummay}}
object has the logic of calculating the remaining quota as follows. Here, the {{directoryCount}}
is considering the snapshots and is causing the {{REM_QUOTA}} evaluated to a negative number,
which is misleading, isn't it?

{code}
      if (quota>0) {
        quotaStr = formatSize(quota, hOption);
        quotaRem = formatSize(quota-(directoryCount+fileCount), hOption);
      }
{code}

Snapshot has the default quota limits. Presently there is no way(command) to set the quota
value for the snapshots.
{code}
DirectorySnapshottableFeature.java

  /** Number of snapshots allowed. */
  private int snapshotQuota = SNAPSHOT_LIMIT;
{code}

> dfs -count -q should not consider snapshots under REM_QUOTA
> -----------------------------------------------------------
>
>                 Key: HDFS-8092
>                 URL: https://issues.apache.org/jira/browse/HDFS-8092
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: snapshots, tools
>            Reporter: Archana T
>            Assignee: Rakesh R
>            Priority: Minor
>
> dfs -count -q should not consider snapshots under Remaining quota
> List of Operations performed-
> 1. hdfs dfs -mkdir /Dir1
> 2. hdfs dfsadmin -setQuota 2 /Dir1
> 3. hadoop fs -count -q -h -v /Dir1
>  
>        QUOTA       {color:red} REM_QUOTA{color}      SPACE_QUOTA REM_SPACE_QUOTA    DIR_COUNT
  FILE_COUNT       CONTENT_SIZE PATHNAME
>            2               {color:red} 1 {color}            none             inf    
       1            0                  0 /Dir1
> 4. hdfs dfs -put hdfs /Dir1/f1
> 5. hadoop fs -count -q -h -v /Dir1
>      QUOTA       {color:red} REM_QUOTA{color}      SPACE_QUOTA REM_SPACE_QUOTA    DIR_COUNT
  FILE_COUNT       CONTENT_SIZE PATHNAME
>            2              {color:red}  0{color}             none             inf    
       1            1             11.4 K /Dir1
> 6. hdfs dfsadmin -allowSnapshot /Dir1
> 7. hdfs dfs -createSnapshot /Dir1
> 8. hadoop fs -count -q -h -v /Dir1
>  QUOTA       {color:red} REM_QUOTA{color}      SPACE_QUOTA REM_SPACE_QUOTA    DIR_COUNT
  FILE_COUNT       CONTENT_SIZE PATHNAME
>            2             {color:red}  -1 {color}            none             inf    
       2            1             11.4 K /Dir1
> Whenever snapshots created the value of REM_QUOTA gets decremented.
> When creation of snaphots are not considered under quota of that respective dir then
dfs -count should not decrement REM_QUOTA value



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message