hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14906) Improvements on FlushLargeStoresPolicy
Date Thu, 03 Dec 2015 06:58:11 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yu Li updated HBASE-14906:
--------------------------
    Fix Version/s: 2.0.0
     Release Note: 
In HBASE-14906 we use "hbase.hregion.memstore.flush.size/column_family_number" as the default
threshold for memstore flush instead of the fixed value through "hbase.hregion.percolumnfamilyflush.size.lower.bound"
property, which makes  the default threshold more flexible to various use case. We also introduce
a new property in name of "hbase.hregion.percolumnfamilyflush.size.lower.bound.min" with 16M
as the default value to avoid small flush in cases like hundreds of column families.

After this change setting "hbase.hregion.percolumnfamilyflush.size.lower.bound" in hbase-site.xml
won't take effect anymore, but expert users could still set this property in table descriptor
to override the default value just as before
           Status: Patch Available  (was: Open)

Add release note and submit patch to see what HadoopQA will say

> Improvements on FlushLargeStoresPolicy
> --------------------------------------
>
>                 Key: HBASE-14906
>                 URL: https://issues.apache.org/jira/browse/HBASE-14906
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>             Fix For: 2.0.0
>
>         Attachments: HBASE-14906.patch, HBASE-14906.v2.patch
>
>
> When checking FlushLargeStoragePolicy, found below possible improving points:
> 1. Currently in selectStoresToFlush, we will do the selection no matter how many actual
families, which is not necessary for one single family
> 2. Default value for hbase.hregion.percolumnfamilyflush.size.lower.bound could not fit
in all cases, and requires user to know details of the implementation to properly set it.
We propose to use "hbase.hregion.memstore.flush.size/column_family_number" instead:
> {noformat}
>   <property>
>     <name>hbase.hregion.percolumnfamilyflush.size.lower.bound</name>
>     <value>16777216</value>
>     <description>
>     If FlushLargeStoresPolicy is used and there are multiple column families,
>     then every time that we hit the total memstore limit, we find out all the
>     column families whose memstores exceed a "lower bound" and only flush them
>     while retaining the others in memory. The "lower bound" will be
>     "hbase.hregion.memstore.flush.size / column_family_number" by default
>     unless value of this property is larger than that. If none of the families
>     have their memstore size more than lower bound, all the memstores will be
>     flushed (just as usual).
>     </description>
>   </property>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message