hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9840) Large scans and BlockCache evictions problems
Date Fri, 25 Oct 2013 06:33:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805088#comment-13805088

Vladimir Rodionov commented on HBASE-9840:

By the way, Lars, 3. would solve the issue you have observed.

> Large scans and BlockCache evictions problems
> ---------------------------------------------
>                 Key: HBASE-9840
>                 URL: https://issues.apache.org/jira/browse/HBASE-9840
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
> I just ran into a scenario that baffled me first, but after some reflection makes sense.
I ran a very large scan that filled up most of the block cache with my scan's data. I ran
that scan a few times.
> That I ran a much smaller scan, and this scan will never get all its blocks cached if
it does not fit entirely into the remaining BlockCache; regardless how I often I run it!
> The reason is that the blocks of the first large scan were all promoted. Since the 2nd
scan did not fully fit into the cache all blocks are round-robin evicted as I rerun the scan.
Thus those blocks will never get accessed more than once before they get evicted again.
> Since promoted blocks are not demoted the large scan's block will never be evicted unless
we have another small enough scan/get that can promote its blocks.
> Not sure what the proper solution is, but it seems only a LRU cache that can expire blocks
over time would solve this.
> Granted, this is a pretty special case.
> Edit: My usual spelling digressions.

This message was sent by Atlassian JIRA

View raw message