hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nijel (JIRA)" <j...@apache.org>
Subject [jira] [Moved] (HDFS-8575) Support User level Quota for space and Name (count)
Date Thu, 11 Jun 2015 08:32:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

nijel moved YARN-3796 to HDFS-8575:
-----------------------------------

        Key: HDFS-8575  (was: YARN-3796)
    Project: Hadoop HDFS  (was: Hadoop YARN)

> Support User level Quota for space and Name (count)
> ---------------------------------------------------
>
>                 Key: HDFS-8575
>                 URL: https://issues.apache.org/jira/browse/HDFS-8575
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: nijel
>            Assignee: nijel
>
> I would like to have one feature in HDFS to have quota management at user level. 
> Background :
> When the customer uses a multi tenant solution it will have many Hadoop eco system components
like HIVE, HBASE, yarn etc. The base folder of these components are different like /hive -
Hive , /hbase -HBase. 
> Now if a user creates some file  or table these will be under the folder specific to
component. If the user name is taken into account it looks like
> {code}
>                                 /hive/user1/table1
>                                 /hive/user2/table1
>                                 /hbase/user1/Htable1
>                                 /hbase/user2/Htable1
>  
> Same for yarn/map-reduce data and logs
> {code}
>  
> In this case restricting the user to use a certain amount of disk/file is very difficult
since the current quota management is at folder level.
>  
> Requirement: User level Quota for space and Name (count). Say user1 can have 100G irrespective
of the folder or location used.
>  
> Here the idea to consider the file owner ad the key and attribute the quota to it.  So
the current quota system can have a initial check for the user quota if defined, before validating
the folder quota.
> Note:
> This need a change in fsimage to store the user and quota information
> Please have a look on this scenario. If it sounds good, i will create the tasks and the
update the design and prototype.
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message