hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Heng Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14556) Make prefetchOnOpen configurable for index and/or data blocks
Date Wed, 14 Oct 2015 07:27:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956420#comment-14956420
] 

Heng Chen commented on HBASE-14556:
-----------------------------------

{quote}
For large datasets it is not viable to load all blocks into memory, but to speed up access
it still makes sense to prefetch the index blocks (being block index and Bloom filter blocks).
{quote}
As current logic,  when readblock,  index blocks will be cached already.  As code below, after
read block from hdfs, it will call {{shouldCacheBlockOnRead}} to decide whether we should
cache the block.
{code}
  public boolean shouldCacheBlockOnRead(BlockCategory category) {
    return isBlockCacheEnabled()
        && (cacheDataOnRead ||
            category == BlockCategory.INDEX ||
            category == BlockCategory.BLOOM ||
            (prefetchOnOpen &&
                (category != BlockCategory.META &&
                 category != BlockCategory.UNKNOWN)));
  }
{code}

> Make prefetchOnOpen configurable for index and/or data blocks
> -------------------------------------------------------------
>
>                 Key: HBASE-14556
>                 URL: https://issues.apache.org/jira/browse/HBASE-14556
>             Project: HBase
>          Issue Type: Bug
>          Components: BlockCache, regionserver
>            Reporter: Lars George
>
> This came up in user discussions. It would be great to add an extra option to the {{CacheConfig}}
that allows to specify what blocks are cached during region/file opening. This should allows
to set {{BlockIndexOnly}}, {{BloomFilterOnly}}, {{AllIndexesOnly}}, {{DataOnly}}, and {{AllBlocks}}.

> For large datasets it is not viable to load all blocks into memory, but to speed up access
it still makes sense to prefetch the index blocks (being block index and Bloom filter blocks).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message