hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy
Date Fri, 04 Dec 2015 02:37:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15039630#comment-15039630
] 

Yu Li commented on HBASE-14906:
-------------------------------

{quote}
So there is no exact lower bound in global config?
{quote}
Yes, there's only a "minimum lower bound" in global config, or say the default value for lower
bound would be "dynamic" and table-specific now instead of any fixed value, unless the auto
computed one is smaller than the minimum config.

After the change there's no way for user to specify a _uniform small_ lower bound for *all*
tables any more, which is also unnecessary IMHO. Expert user who wants to specify a smaller
lower bound for any specified table could still achieve the goal by setting {{hbase.hregion.percolumnfamilyflush.size.lower.bound}}
in table descriptor

Feel free to let me know if any concern here. Thanks.

> Improvements on FlushLargeStoresPolicy
> --------------------------------------
>
>                 Key: HBASE-14906
>                 URL: https://issues.apache.org/jira/browse/HBASE-14906
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>             Fix For: 2.0.0
>
>         Attachments: HBASE-14906.patch, HBASE-14906.v2.patch, HBASE-14906.v3.patch, HBASE-14906.v4.patch,
HBASE-14906.v4.patch
>
>
> When checking FlushLargeStoragePolicy, found below possible improving points:
> 1. Currently in selectStoresToFlush, we will do the selection no matter how many actual
families, which is not necessary for one single family
> 2. Default value for hbase.hregion.percolumnfamilyflush.size.lower.bound could not fit
in all cases, and requires user to know details of the implementation to properly set it.
We propose to use "hbase.hregion.memstore.flush.size/column_family_number" instead:
> {noformat}
>   <property>
>     <name>hbase.hregion.percolumnfamilyflush.size.lower.bound</name>
>     <value>16777216</value>
>     <description>
>     If FlushLargeStoresPolicy is used and there are multiple column families,
>     then every time that we hit the total memstore limit, we find out all the
>     column families whose memstores exceed a "lower bound" and only flush them
>     while retaining the others in memory. The "lower bound" will be
>     "hbase.hregion.memstore.flush.size / column_family_number" by default
>     unless value of this property is larger than that. If none of the families
>     have their memstore size more than lower bound, all the memstores will be
>     flushed (just as usual).
>     </description>
>   </property>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message