hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashu Pachauri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16755) Honor flush policy under global memstore pressure
Date Thu, 02 Mar 2017 18:44:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892760#comment-15892760

Ashu Pachauri commented on HBASE-16755:

And there is another minor reason why we set force to true is that, we select regions based
on their total memstore size. If we do not flush all the contents in memstore, then the best
candidate may not be the one which has the maximum memstore size.
Maybe we could introduce new methods in FlushPolicy to find out the lfushable size and select
regions based on that value. Of course this is only a nice to have, can do it later if anyone
has interest.
Makes sense.

bq. On the patch, as now we rely on the FlushPolicy to always return something to flush, maybe
we should add some checks in the code? We may introduce new flush policies in the future.
we need to make sure that they also follow the rule.
A flush policy that returns nothing to flush (given that the decision whether a region is
flushable is made before querying the flush policy) sounds impractical to use. But, to protect
against such scenarios, we can either pass down the "emergencyFlush" flag to the Region#flush
which resorts to flushing all store (if flush policy returns none) in case of an emergency
flush. Another option could be to return FlushResult.Result.CANNOT_FLUSH from HRegion#flushCache
if flush policy returns no stores to flush for the region and the caller (MemstoreFlusher)
can decide to retry with forceFlushAllStores under certain circumstances.

> Honor flush policy under global memstore pressure
> -------------------------------------------------
>                 Key: HBASE-16755
>                 URL: https://issues.apache.org/jira/browse/HBASE-16755
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Ashu Pachauri
>            Assignee: Ashu Pachauri
>             Fix For: 1.3.1
>         Attachments: HBASE-16755.v0.patch
> When global memstore reaches the low water mark, we pick the best flushable region and
flush all column families for it. This is a suboptimal approach in the  sense that it leads
to an unnecessarily high file creation rate and IO amplification due to compactions. We should
still try to honor the underlying FlushPolicy.

This message was sent by Atlassian JIRA

View raw message