hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhilash (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-13876) Improving performance of HeapMemoryManager
Date Tue, 09 Jun 2015 21:19:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Abhilash updated HBASE-13876:
-----------------------------
    Description: 
I am trying to improve the performance of DefaultHeapMemoryTuner by introducing some more
checks. The current checks under which the DefaultHeapMemoryTuner works are very rare so I
am trying to weaken these checks to improve its performance.
Check current memstore size and current block cache size. If we are using less than 50% of
currently available block cache size  we say block cache is sufficient and same for memstore.
This check will be very effective when cluster is load heavy or write heavy. Earlier version
just waited to for number of evictions / number of flushes to be zero which is very rare to
happen.
Otherwise based on percent change in number of cache misses and number of flushes we increase
/ decrease memory provided for caching / memstore. After doing so, on next call of HeapMemoryTuner
we verify that last change has indeed decreased number of evictions / flush ( combined). I
am doing this analysis by comparing percent change (which is basically nothing but normalized
of the derivative) of number of evictions and number of flushes in last two periods. The main
motive for doing this was that if we have random reads then even after increasing block cache
we wont be able to decrease number of cache misses and eventually we will not waste memory
on block caches.
  

  was:I am trying to improve the performance of DefaultHeapMemoryTuner by introducing some
more checks. The current checks under which the DefaultHeapMemoryTuner works are very rare
so I am trying to weaken these checks to improve its performance.


> Improving performance of HeapMemoryManager
> ------------------------------------------
>
>                 Key: HBASE-13876
>                 URL: https://issues.apache.org/jira/browse/HBASE-13876
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbase, regionserver
>    Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
>            Reporter: Abhilash
>            Assignee: Abhilash
>            Priority: Minor
>         Attachments: HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by introducing some
more checks. The current checks under which the DefaultHeapMemoryTuner works are very rare
so I am trying to weaken these checks to improve its performance.
> Check current memstore size and current block cache size. If we are using less than 50%
of currently available block cache size  we say block cache is sufficient and same for memstore.
This check will be very effective when cluster is load heavy or write heavy. Earlier version
just waited to for number of evictions / number of flushes to be zero which is very rare to
happen.
> Otherwise based on percent change in number of cache misses and number of flushes we
increase / decrease memory provided for caching / memstore. After doing so, on next call of
HeapMemoryTuner we verify that last change has indeed decreased number of evictions / flush
( combined). I am doing this analysis by comparing percent change (which is basically nothing
but normalized of the derivative) of number of evictions and number of flushes in last two
periods. The main motive for doing this was that if we have random reads then even after increasing
block cache we wont be able to decrease number of cache misses and eventually we will not
waste memory on block caches.
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message