Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 6 Jan 2014 18:11:50 +0000 (UTC)
From: "Nick Dimiduk (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12686841.1388492059471.48149.1389031910573@arcas>
In-Reply-To: <JIRA.12686841.1388492059471@arcas>
References: <JIRA.12686841.1388492059471@arcas>
Subject: [jira] [Commented] (HBASE-10263) make LruBlockCache
 single/multi/in-memory ratio user-configurable and provide preemptive mode
 for in-memory type block
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13863181#comment-13863181 ] 

Nick Dimiduk commented on HBASE-10263:
--------------------------------------

Evictions happen on a background thread. Filling the cache and then immediately checking the eviction count results in a race between the current thread and the eviction thread; thus this is very likely a flakey test on our over-extended build machines.

{noformat}
+    // 5th single block
+    cache.cacheBlock(singleBlocks[4].cacheKey, singleBlocks[4]);
+    expectedCacheSize += singleBlocks[4].cacheBlockHeapSize();
+    // Do not expect any evictions yet
+    assertEquals(0, cache.getEvictionCount());
+    // Verify cache size
+    assertEquals(expectedCacheSize, cache.heapSize());
{noformat}

In the above block, the call to cacheBlock() will only notify the eviction thread, not force eviction. A yield or short sleep should be inserted before the call to getEvictionCount() in order to help reduce the chance of exercising the race condition. Repeat for all the following stanzas.

> make LruBlockCache single/multi/in-memory ratio user-configurable and provide preemptive mode for in-memory type block
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-10263
>                 URL: https://issues.apache.org/jira/browse/HBASE-10263
>             Project: HBase
>          Issue Type: Improvement
>          Components: io
>            Reporter: Feng Honghua
>            Assignee: Feng Honghua
>         Attachments: HBASE-10263-trunk_v0.patch, HBASE-10263-trunk_v1.patch
>
>
> currently the single/multi/in-memory ratio in LruBlockCache is hardcoded 1:2:1, which can lead to somewhat counter-intuition behavior for some user scenario where in-memory table's read performance is much worse than ordinary table when two tables' data size is almost equal and larger than regionserver's cache size (we ever did some such experiment and verified that in-memory table random read performance is two times worse than ordinary table).
> this patch fixes above issue and provides:
> 1. make single/multi/in-memory ratio user-configurable
> 2. provide a configurable switch which can make in-memory block preemptive, by preemptive means when this switch is on in-memory block can kick out any ordinary block to make room until no ordinary block, when this switch is off (by default) the behavior is the same as previous, using single/multi/in-memory ratio to determine evicting.
> by default, above two changes are both off and the behavior keeps the same as before applying this patch. it's client/user's choice to determine whether or which behavior to use by enabling one of these two enhancements.


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)