hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allan Yang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
Date Wed, 22 Mar 2017 03:09:41 GMT

     [ https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Allan Yang updated HBASE-17757:
    Release Note: Blocksize is set in columnfamily's atrributes. It is used to control block
sizes when generating blocks. But, it doesn't take encoding into count. If you set encoding
to blocks, after encoding, the block size varies. Since blocks will be cached in memory after
encoding (default), it will cause memory fragment if using blockcache, or decrease the pool
efficiency if using bucketCache. This issue introduced a new config named 'hbase.writer.unified.encoded.blocksize.ratio'.
The default value of this config is 1, meaning doing nothing. If this value is set to a smaller
value like 0.5, and the blocksize is set to 64KB(default value of blocksize). It will unify
the blocksize after encoding to 64KB * 0.5 = 32KB. Unified blocksize will releaf the memory
problems mentioned above.

> Unify blocksize after encoding to decrease memory fragment 
> -----------------------------------------------------------
>                 Key: HBASE-17757
>                 URL: https://issues.apache.org/jira/browse/HBASE-17757
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Allan Yang
>            Assignee: Allan Yang
>         Attachments: HBASE-17757.patch, HBASE-17757v2.patch
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache. Though we have
set the blocksize, after encoding, blocksize is varied. Varied blocksize will cause memory
fragment problem, which will result in more FGC finally.In order to relief the memory fragment,
This issue adjusts the encoded block to a unified size.

This message was sent by Atlassian JIRA

View raw message