hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-9840) Large scans and BlockCache evictions problems
Date Fri, 25 Oct 2013 05:41:30 GMT
Lars Hofhansl created HBASE-9840:

             Summary: Large scans and BlockCache evictions problems
                 Key: HBASE-9840
                 URL: https://issues.apache.org/jira/browse/HBASE-9840
             Project: HBase
          Issue Type: Bug
            Reporter: Lars Hofhansl

I just ran into a scenario that baffled me first, but after some reflection makes sense. I
ran a very large scan that filled up most of the block cache with my scan's data. I ran that
scan a few times.

That I ran a smaller scan, and this scan will never get all its blocks cached if it does not
fit into the remaining BlockCache, regardless how I often I run it.

The reason is that the block of the first large scan were all promoted. Since the 2nd scan
did not fully fit into the cache all block a round-robin evicted as I rerun the scan, and
those blocks will never get accessed more than once before they get evicted again.

Since promoted blocks are not demoted the large scan's block will never be evicted unless
we have another small enough scan that can promote its blocks.

Not sure what the proper solution is, but it seems only a LRU cache that can expire blocks
over time would solve.

Granted, this is a pretty special case.

This message was sent by Atlassian JIRA

View raw message