hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1026) Quota checks fail for small files and quotas
Date Tue, 29 Jul 2014 23:54:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078639#comment-14078639
] 

Hadoop QA commented on HDFS-1026:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12520273/HDFS-1026.pacth
  against trunk revision .

    {color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7490//console

This message is automatically generated.

> Quota checks fail for small files and quotas
> --------------------------------------------
>
>                 Key: HDFS-1026
>                 URL: https://issues.apache.org/jira/browse/HDFS-1026
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: documentation, namenode
>    Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0
>            Reporter: Eli Collins
>              Labels: newbie
>         Attachments: HDFS-1026.pacth
>
>
> If a directory has a quota less than blockSize * numReplicas then you can't add a file
to it, even if the file size is less than the quota. This is because FSDirectory#addBlock
updates the count assuming at least one block is written in full. We don't know how much of
the block will be written when addBlock is called and supporting such small quotas is not
important so perhaps we should document this and log an error message instead of making small
(blockSize * numReplicas) quotas work.
> {code}
> // check quota limits and updated space consumed
> updateCount(inodes, inodes.length-1, 0, fileINode.getPreferredBlockSize()*fileINode.getReplication(),
true);
> {code}
> You can reproduce with the following commands:
> {code}
> $ dd if=/dev/zero of=temp bs=1000 count=64
> $ hadoop fs -mkdir /user/eli/dir
> $ hdfs dfsadmin -setSpaceQuota 191M /user/eli/dir
> $ hadoop fs -put temp /user/eli/dir  # Causes DSQuotaExceededException
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message