hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
Date Mon, 13 Mar 2017 23:19:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923165#comment-15923165

Anoop Sam John commented on HBASE-17757:

if((unifyEncodedBlockSizeEnabled && blockWriter.encodedBlockSizeWritten() >= encodedBlockSizeLimit)
327	        || (blockWriter.blockSizeWritten() >= hFileContext.getBlocksize())) {
This is what there in code. So this is an || check.  I was thinking of same way only when
having a ratio only config.  The boolean enable/disable config , with defaults to disable
is not required if we make ratio = 1.  That means the check will be like 
blockWriter.encodedBlockSizeWritten() >= encodedBlockSizeLimit || blockWriter.blockSizeWritten()
>= hFileContext.getBlocksize()  so only the old way of block size determines the block
boundary.  Ya encodeBlockSize new config also same way. Rather than say a ratio, say exact
size. Both way almost same. 

> Unify blocksize after encoding to decrease memory fragment 
> -----------------------------------------------------------
>                 Key: HBASE-17757
>                 URL: https://issues.apache.org/jira/browse/HBASE-17757
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Allan Yang
>            Assignee: Allan Yang
>         Attachments: HBASE-17757.patch
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache. Though we have
set the blocksize, after encoding, blocksize is varied. Varied blocksize will cause memory
fragment problem, which will result in more FGC finally.In order to relief the memory fragment,
This issue adjusts the encoded block to a unified size.

This message was sent by Atlassian JIRA

View raw message