hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
Date Fri, 13 Jun 2014 18:33:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030968#comment-14030968
] 

Nick Dimiduk commented on HBASE-11331:
--------------------------------------

In this implementation, the decompressed block does not replace the compressed block in the
cache. Decompression cost is paid on block access, every time. I need to profile the scanner
path to ensure a single request is not decompressing the same block multiple times. For hot
blocks, I expect this to result in increased CPU load vs decompressing it only once. For a
more evenly distributed access pattern, this should greatly reduce the amount of disk seeks
because more data is cached. I believe the latter use-case is more common.

> [blockcache] lazy block decompression
> -------------------------------------
>
>                 Key: HBASE-11331
>                 URL: https://issues.apache.org/jira/browse/HBASE-11331
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>         Attachments: HBASE-11331.00.patch
>
>
> Maintaining data in its compressed form in the block cache will greatly increase our
effective blockcache size and should show a meaning improvement in cache hit rates in well
designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed,
rather than as soon as they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message