hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3938) Quotas for disk space management
Date Fri, 19 Sep 2008 23:46:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632890#action_12632890
] 

Raghu Angadi commented on HADOOP-3938:
--------------------------------------

How to test with quota violations in fsimage :
===================================

* Comment out two places where {{QuotaExceededException}} is thrown in {{INodeDirectoryWithQuota}}
* Run a fresh HDFS and create a few files and directories such that it exceeds quotas for
a few directories. For e.g. : {noformat}
bin/hadoop fs -mkdir quota3-3k
bin/hadoop dfsadmin -setQuota 3 quota3-3k
bin/hadoop fs -mkdir quota3-3k/emptyDir
bin/hadoop fs -put /dev/null quota3-3k/emptyFile
bin/hadoop dfsadmin -setSpaceQuota 3072 quota3-3k
bin/hadoop fs -put ~/cws/tmp/5Mb quota3-3k/

bin/hadoop fs -mkdir quota3-3k/quota2
bin/hadoop dfsadmin -setQuota 2 quota3-3k/quota2
bin/hadoop fs -mkdir quota3-3k/quota2/emptyDir
bin/hadoop fs -put /dev/null quota3-3k/quota2/emptyFile

bin/hadoop fs -mkdir quota3-3k/quota-5k
bin/hadoop dfsadmin -setSpaceQuota 5120 quota3-3k/quota-5k
bin/hadoop fs -put ~/cws/tmp/5Mb quota3-3k/quota-5k
bin/hadoop fs -mkdir quota3-3k/quota-5k/emptyDir
{noformat}
* Stop the cluster and start it again.
* The cluster should restart fine. You should see some warnings for {{quota3-3k}} etc in NameNode
log.
* Adding any more files or data to such directories should fail.
* You should be able to either increase the quotas or delete some files to satisfy the requirements.
* Once these are fixed, there should not be any more warnings when NameNode restarts.
 

> Quotas for disk space management
> --------------------------------
>
>                 Key: HADOOP-3938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3938
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: Robert Chansler
>            Assignee: Raghu Angadi
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3938.patch, HADOOP-3938.patch, HADOOP-3938.patch, HADOOP-3938.patch,
HADOOP-3938.patch, HADOOP-3938.patch, HADOOP-3938.patch, HADOOP-3938.patch, HADOOP-3938.patch,
hdfs_quota_admin_guide.pdf, hdfs_quota_admin_guide.xml
>
>
> Directory quotas for bytes limit the number of bytes used by files in and below the directory.
Operation is independent of name quotas (HADOOP-3187), but the implementation is parallel.
Each file is charged according to its length multiplied by its intended replication factor.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message