hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
Date Mon, 25 Aug 2014 21:57:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109817#comment-14109817

Nick Dimiduk commented on HBASE-11331:

bq. Would be interesting to see how it does in a long running test?

Today's results were from 90m warmup + 40m test. The 16h run I posted on Thursday [0] seemed
stable. No slope in those graphs. This is just PE mind you, no concurrent writes.

bq. On the blockCacheCount, this is indication of our caching more blocks? 3x?

Yes, exactly. It's purely compression ratio manifest in memory. All this is using SNAPPY and
the reported compression ratio is 0.2473, thus the ~3x increase. Probably using GZ would have
better BC utilization buy higher CPU load (should translate to a standard compression benchmark).

[0]: https://issues.apache.org/jira/secure/attachment/12663479/v03-20g-045g-true-16h.pdf

> [blockcache] lazy block decompression
> -------------------------------------
>                 Key: HBASE-11331
>                 URL: https://issues.apache.org/jira/browse/HBASE-11331
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>         Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, HBASE-11331.02.patch,
HBASE-11331.03.patch, HBASE-11331.04.patch, HBASE-11331.05.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf,
hbase-hbase-master-hor17n36.gq1.ygridcore.net.log, lazy-decompress.02.0.pdf, lazy-decompress.02.1.json,
lazy-decompress.02.1.pdf, v03-20g-045g-false.pdf, v03-20g-045g-true-16h.pdf, v03-20g-045g-true.pdf
> Maintaining data in its compressed form in the block cache will greatly increase our
effective blockcache size and should show a meaning improvement in cache hit rates in well
designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed,
rather than as soon as they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.

This message was sent by Atlassian JIRA

View raw message