hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhilash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13876) Improving performance of HeapMemoryManager
Date Wed, 10 Jun 2015 17:24:02 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580846#comment-14580846

Abhilash commented on HBASE-13876:

Thanks a lot for the reviews.
Going to use ENUM and introducing NEUTRAL step to remove unnecessary tuning. Which will have
better use of the final variable "tolerance".
Thinking about how to remove constants like 50%. Possibly provide a static final member to
adjust the range when we say memstore/block cache is sufficient or not.
I was using current value of number of evictCount and number of flushes because they are non
zero if program enter that block but previous evictCount and flushes may be zero. Thinking
about how to use previous values without introducing lots of checks.

> Improving performance of HeapMemoryManager
> ------------------------------------------
>                 Key: HBASE-13876
>                 URL: https://issues.apache.org/jira/browse/HBASE-13876
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbase, regionserver
>    Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
>            Reporter: Abhilash
>            Assignee: Abhilash
>            Priority: Minor
>         Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch, HBASE-13876.patch
> I am trying to improve the performance of DefaultHeapMemoryTuner by introducing some
more checks. The current checks under which the DefaultHeapMemoryTuner works are very rare
so I am trying to weaken these checks to improve its performance.
> Check current memstore size and current block cache size. If we are using less than 50%
of currently available block cache size  we say block cache is sufficient and same for memstore.
This check will be very effective when server is either load heavy or write heavy. Earlier
version just waited for number of evictions / number of flushes to be zero which are very
> Otherwise based on percent change in number of cache misses and number of flushes we
increase / decrease memory provided for caching / memstore. After doing so, on next call of
HeapMemoryTuner we verify that last change has indeed decreased number of evictions / flush
( combined). I am doing this analysis by comparing percent change (which is basically nothing
but normalized derivative) of number of evictions and number of flushes during last two periods.
The main motive for doing this was that if we have random reads then we will be having a lot
of cache misses. But even after increasing block cache we wont be able to decrease number
of cache misses and we will revert back and eventually we will not waste memory on block caches.
This will also help us ignore random short term spikes in reads / writes.

This message was sent by Atlassian JIRA

View raw message