hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16650) Wrong usage of BlockCache eviction stat for heap memory tuning
Date Wed, 21 Sep 2016 02:30:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508478#comment-15508478

Anoop Sam John commented on HBASE-16650:

Thanks Stack
bq.Is it right putting code under this check: if (evictedByEvictionProcess) {
The code was already under this check..  What I did is moved the stat update also inside the
bq.We are setting some flags in this block that are used later. Do we need those flags when
an eviction because an hfile has been removed?
Sorry where is this set?
bq.This is ugly but we are already keeping blockcache in a global.... when we fix this, we
can fix this new addition:
Ya we need this new as u can see we pass the L1 cache instance to HeapMemManager so that part
we init L1 cache alone and later some one init global cache it has to use this same instance.

> Wrong usage of BlockCache eviction stat for heap memory tuning
> --------------------------------------------------------------
>                 Key: HBASE-16650
>                 URL: https://issues.apache.org/jira/browse/HBASE-16650
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 1.0.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0, 1.4.0
>         Attachments: HBASE-16650.patch
> 1. We use the stat evictedBlocksCount - A block can get evicted because of eviction thread
due to lack of space or because of removal of an HFile itself (After a compaction). We should
not consider the latter in the tune decision at all. These are actually invalidation of blocks.
Should the stat counter itself not use this count of evicted blocks? I think yes. This will
give wrong message to users that there are lot of real eviction happening.
> 2. In case L1+ L2 combined block cache, what we use is the sum of evictions from both.
But we will be tuning L1 size alone. Eviction count from L2 should not affect the tuning of

This message was sent by Atlassian JIRA

View raw message