hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhihong Yu (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5349) Automagically tweak global memstore and block cache sizes based on workload
Date Thu, 09 Feb 2012 07:29:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204325#comment-13204325
] 

Zhihong Yu commented on HBASE-5349:
-----------------------------------

I am thinking of introducing rolling counters (using circular buffer) for both MemStore and
LruBlockCache.
We record the number of flushes (for MemStore) and evictions (for LruBlockCache), respectively.
Every 1 (configurable) minute, we roll the counters.
It should be straight forward to observe whether MemStore or LruBlockCache is under pressure
by looking at the trend of the rolling counters.
Once we determine the one under pressure, we can utilize what J-D described above to shift
heap among the two.

We can also introduce weights between MemStore and LruBlockCache for the rolling counters.


                
> Automagically tweak global memstore and block cache sizes based on workload
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-5349
>                 URL: https://issues.apache.org/jira/browse/HBASE-5349
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.92.0
>            Reporter: Jean-Daniel Cryans
>             Fix For: 0.94.0
>
>
> Hypertable does a neat thing where it changes the size given to the CellCache (our MemStores)
and Block Cache based on the workload. If you need an image, scroll down at the bottom of
this link: http://www.hypertable.com/documentation/architecture/
> That'd be one less thing to configure.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message