hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu Shaohui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15338) Add a option to disable the data block cache for testing the performance of underlying file system
Date Fri, 26 Feb 2016 06:29:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15168533#comment-15168533
] 

Liu Shaohui commented on HBASE-15338:
-------------------------------------

[~anoop.hbase]
{quote}
ut my thinking was when we specify cache blocks as false in Get/Scan, we will consider that
for DATA blocks only. The index blocks will get cached any way. Did not see the code now..
Can check
{quote}
After the check the code in HFileReaderImpl.java#1533, the index blocks will be also not cached
when we specify cache blocks as false in Get/Scan, because the var cacheBlock is false.
What's more, we don't like to change the client code when using 3rd-party benchmark tools
like YCSB.
{code}
        // Cache the block if necessary
        if (cacheBlock && cacheConf.shouldCacheBlockOnRead(category)) {
          cacheConf.getBlockCache().cacheBlock(cacheKey,
            cacheConf.shouldCacheCompressed(category) ? hfileBlock : unpacked,
            cacheConf.isInMemory(), this.cacheConf.isCacheDataInL1());
        }
{code}

> Add a option to disable the data block cache for testing the performance of underlying
file system
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15338
>                 URL: https://issues.apache.org/jira/browse/HBASE-15338
>             Project: HBase
>          Issue Type: Improvement
>          Components: integration tests
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15338-trunk-v1.diff
>
>
> When testing and comparing the performance of different file systems(HDFS, Azure blob
storage, AWS S3 and so on) for HBase, it's better to avoid the affect of the HBase BlockCache
and get the actually random read latency when data block is read from underlying file system.
(Usually, the index block and meta block should be cached in memory in the testing).
> So we add a option in CacheConfig to disable the data block cache.
> Suggestions are welcomed~ Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message