hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3976) Disable Block Cache On Compactions
Date Wed, 29 Aug 2012 18:04:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444264#comment-13444264

Lars Hofhansl commented on HBASE-3976:

In a typical read/write type load you'd want "hbase.rs.cacheblocksonwrite" on, no?
The data in the memstore is the newest, which will typically be the most interesting data
in the future. Without "hbase.rs.cacheblocksonwrite" a memflush will make that data cold and
it needs to be loaded into the cache upon the next set of reads.
The same is actually true (to a lesser extend) for compactions: We take the latest data and
make it cold.

"hbase.rs.cacheblocksonwrite" actually does not mean much, what writes are we talking about?
I can see see three different options here: (1) cache on flush, (2) cache on minor compaction,
and *maybe* (3) cache on major compaction... Maybe that's overkill (especially the last one)?

> Disable Block Cache On Compactions
> ----------------------------------
>                 Key: HBASE-3976
>                 URL: https://issues.apache.org/jira/browse/HBASE-3976
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.90.3
>            Reporter: Karthick Sankarachary
>            Assignee: Mikhail Bautin
>            Priority: Minor
>         Attachments: HBASE-3976.patch, HBASE-3976-unconditional.patch, HBASE-3976-V3.patch
> Is there a good reason to believe that caching blocks during compactions is beneficial?
Currently, if block cache is enabled on a certain family, then every time it's compacted,
we load all of its blocks into the (LRU) cache, at the expense of the legitimately hot ones.
> As a matter of fact, this concern was raised earlier in HBASE-1597, which rightly points
out that, "we should not bog down the LRU with unneccessary blocks" during compaction. Even
though that issue has been marked as "fixed", it looks like it ought to be reopened.
> Should we err on the side of caution and not cache blocks during compactions period (as
illustrated in the attached patch)? Or, can we be selectively aggressive about what blocks
do get cached during compaction (e.g., only cache those blocks from the recent files)?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message