hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy
Date Thu, 03 Dec 2015 02:53:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037148#comment-15037148

Duo Zhang commented on HBASE-14906:

About config in table description, I think we need to preserve a way for experts to override
the default value, in case they really need to set a smaller value than region.getMemstoreFlushSize()
/ familyNumber for their use case
Sounds reasonable. What about introduce two configurations here? One is the lower bound of
lower bound, one is the exact lower bound? Make global config and table config have different
meanings is a little confusing I think.

> Improvements on FlushLargeStoresPolicy
> --------------------------------------
>                 Key: HBASE-14906
>                 URL: https://issues.apache.org/jira/browse/HBASE-14906
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-14906.patch
> When checking FlushLargeStoragePolicy, found below possible improving points:
> 1. Currently in selectStoresToFlush, we will do the selection no matter how many actual
families, which is not necessary for one single family
> 2. Default value for hbase.hregion.percolumnfamilyflush.size.lower.bound could not fit
in all cases, and requires user to know details of the implementation to properly set it.
We propose to use "hbase.hregion.memstore.flush.size/column_family_number" instead:
> {noformat}
>   <property>
>     <name>hbase.hregion.percolumnfamilyflush.size.lower.bound</name>
>     <value>16777216</value>
>     <description>
>     If FlushLargeStoresPolicy is used and there are multiple column families,
>     then every time that we hit the total memstore limit, we find out all the
>     column families whose memstores exceed a "lower bound" and only flush them
>     while retaining the others in memory. The "lower bound" will be
>     "hbase.hregion.memstore.flush.size / column_family_number" by default
>     unless value of this property is larger than that. If none of the families
>     have their memstore size more than lower bound, all the memstores will be
>     flushed (just as usual).
>     </description>
>   </property>
> {noformat}

This message was sent by Atlassian JIRA

View raw message