hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-16650) Wrong usage of BlockCache eviction stat for heap memory tuning
Date Thu, 22 Sep 2016 16:03:20 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anoop Sam John updated HBASE-16650:
-----------------------------------
      Resolution: Fixed
    Hadoop Flags: Incompatible change,Reviewed
    Release Note: Changed tracking of evictedBlocks count NOT to include evictions of blocks
for a removed HFile. HFiles gets removed after compaction
          Status: Resolved  (was: Patch Available)

Pushed to master. Thanks for the review Stack.
Though it is applicable in 1.x branches also, not committing.  HeapMemory tuning is default
ON there.  This is part of making it ready for turning ON by default.

> Wrong usage of BlockCache eviction stat for heap memory tuning
> --------------------------------------------------------------
>
>                 Key: HBASE-16650
>                 URL: https://issues.apache.org/jira/browse/HBASE-16650
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 1.0.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-16650.patch, HBASE-16650_V2.patch, HBASE-16650_V3.patch, HBASE-16650_V4.patch,
HBASE-16650_V5.patch
>
>
> 1. We use the stat evictedBlocksCount - A block can get evicted because of eviction thread
due to lack of space or because of removal of an HFile itself (After a compaction). We should
not consider the latter in the tune decision at all. These are actually invalidation of blocks.
Should the stat counter itself not use this count of evicted blocks? I think yes. This will
give wrong message to users that there are lot of real eviction happening.
> 2. In case L1+ L2 combined block cache, what we use is the sum of evictions from both.
But we will be tuning L1 size alone. Eviction count from L2 should not affect the tuning of
L1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message