hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7658) HDFS Space Quota not working as expected
Date Thu, 22 Jan 2015 09:31:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287166#comment-14287166
] 

Vinayakumar B commented on HDFS-7658:
-------------------------------------


bq. Then i tried to put a file of 100 Mb . It should since it will just consume 200 Mb of
space with replication. But when i put that i got an error
HDFS doesnt know the complete size of the file ahead. It considers default blocksize ( in
your case 256MB) for calculation while adding the new block.
So with RF 2, always quota should be more than 512 MB in your case to upload the file successfully.

With this, I am surprised how you are able to upload first 2 files.?

Did you uploaded to some other directory and moved to this dir? 
or set the quota after uploading these 2 files.?

> HDFS Space Quota not working as expected
> ----------------------------------------
>
>                 Key: HDFS-7658
>                 URL: https://issues.apache.org/jira/browse/HDFS-7658
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: CDH4.6
>            Reporter: Puttaswamy
>
> I am implementing hdfs quota in a cdh4.6 cluster .Hdfs name quota has been working properly.But
the Hdfs Space quota has not been working as expected.i.e,
> I set the space quota of 500MB for a directory say /test-space-quota.
> Then i put a file of 10 Mb into /test-space-quota which worked .Now the space available
is 480 MB ( 500 - 10*2) where 2 is rep factor.
> Then i put a file of 50Mb into /test-space-quota which worked too as expected. Now the
space available is 380 MB (480 - 50*2)
> "I am checking the quota left from the command hadoop fs -count -q /test-space-quota"
> Then i tried to put a file of 100 Mb . It should since it will just consume 200 Mb of
space with replication. But when i put that i got an error 
> "DataStreamer Exception
> org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /test
is exceeded: quota = 524288000 B = 500 MB but diskspace consumed = 662700032 B = 632 MB"
> But the quota says
> hadoop fs -count -q /test-space-quota
>         none             inf       524288000       398458880            1           
2           62914560 /test-space-quota
> Could you please help on this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message