hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhilash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13980) Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
Date Mon, 29 Jun 2015 23:05:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606617#comment-14606617
] 

Abhilash commented on HBASE-13980:
----------------------------------

In my opinion rather than giving more weight (it would be difficult in itself to reason about
their relative weights) to blockedFlushCount. We should directly take tuning decision if we
observe blocked flushes in steady state. Because they are rare and very undesirable.
One complication is that when memory tuner increases block cache size and decreases memstore
size. It might force many blocked flushes but that does not mean we have to abruptly increase
 memstore size. So we should do that only in steady state. 
If after reading this you have doubt about how tuner would works then ?
Ans : Suppose in a certain period tuner(1st tuning) increased block cache by 8% and decreased
memstore by same amount. Then in next step(if server is not completely read heavy, if it is
ready heavy then we are good anyways) we will observe many blocked / unblocked flushes. Then
if this is number is really very high compared to past trend we revert our tuning. But not
completely, we partially revert previous tuning operation(2nd tuning). Overall we have increased
block cache by 4% which tuner wanted to do in 1st tuning. 

> Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
> ----------------------------------------------------------------------------
>
>                 Key: HBASE-13980
>                 URL: https://issues.apache.org/jira/browse/HBASE-13980
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Abhilash
>            Priority: Minor
>
> Currently DefaultHeapMemoryTuner doesn't distinguish blockedFlushCount vs unblockedFlushCount.
> In its tune() method:
> {code}
>     long totalFlushCount = blockedFlushCount+unblockedFlushCount;
>     rollingStatsForCacheMisses.insertDataValue(cacheMissCount);
>     rollingStatsForFlushes.insertDataValue(totalFlushCount);
> {code}
> Occurrence of blocked flush indicates that upper limit for memstore is not sufficient.
> We should either give blockedFlushCount more weight or, take tuning action based on blockedFlushCount
directly.
> See discussion from tail of HBASE-13876.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message