hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: [DISCUSS] FileSystem Quotas in HBase
Date Wed, 02 Nov 2016 19:15:57 GMT
Thanks for the reviews so far, Ted and Stack. The comments were great 
and much appreciated.

Interpreting consensus from lack of objection, I'm going to move ahead 
in earnest starting to work on what was described in the doc. Expect to 
see some work break-out happening under HBASE-16961 and patches starting 
to land.

I'm also happy to entertain more discussion if anyone hasn't found the 
time to read/comment yet.

Thanks!

- Josh

Josh Elser wrote:
> Sure thing, Ted.
>
> https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing
>
>
> Let me open an umbrella issue for now. I can break up the work later.
>
> https://issues.apache.org/jira/browse/HBASE-16961
>
> Ted Yu wrote:
>> Josh:
>> Can you put the doc in google doc so that people can comment on it ?
>>
>> Is there a JIRA opened for this work ?
>> Please open one if there is none.
>>
>> Thanks
>>
>> On Fri, Oct 28, 2016 at 9:00 AM, Josh Elser<elserj@apache.org> wrote:
>>
>>> Hi folks,
>>>
>>> I'd like to propose the introduction of FileSystem quotas to HBase.
>>>
>>> Here's a design doc[1] available which (hopefully) covers all of the
>>> salient points of what I think an initial version of such a feature
>>> would
>>> include.
>>>
>>> tl;dr We can define quotas on tables and namespaces. Region size is
>>> computed by RegionServers and sent to the Master. The Master inspects
>>> the
>>> sizes of Regions, rolling up to table and namespace sizes. Defined
>>> quotas
>>> in the quota table are evaluated given the computed sizes, and, for
>>> those
>>> tables/namespaces violating the quota, RegionServers are informed to
>>> take
>>> some action to limit any further filesystem growth by that
>>> table/namespace.
>>>
>>> I'd encourage you to give the document a read -- I tried to cover as
>>> much
>>> as I could without getting unnecessarily bogged down in implementation
>>> details.
>>>
>>> Feedback is, of course, welcomed. I'd like to start sketching out a
>>> breakdown of the work (all writing and no programming makes Josh a sad
>>> boy). I'm happy to field any/all questions. Thanks in advance.
>>>
>>> - Josh
>>>
>>> [1] http://home.apache.org/~elserj/hbase/FileSystemQuotasforApac
>>> heHBase.pdf
>>>
>>

Mime
View raw message