hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17757) Unify blocksize after encoding to decrease memory fragment
Date Wed, 08 Mar 2017 07:51:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900866#comment-15900866
] 

Anoop Sam John edited comment on HBASE-17757 at 3/8/17 7:51 AM:
----------------------------------------------------------------

In case where both DBE and compression in use, the size we track will be after compression
also. And what we keep in cache is uncompressed blocks (By default. There is config to keep
compressed also)..  So the math will go wrong there?
Now also the block sizes for BucketCache has to be tuned considering the encoding factor.
 Ya it makes the size restrictions bit more clear.  So when there is compression also in place,
need to adjust the factor accordingly say 0.5 to 0.3 (eg: )


was (Author: anoop.hbase):
In case where both DBE and compression in use, the size we track will be after compression
also. And what we keep in cache is uncompressed blocks (By default. There is config to keep
compressed also)..  So the math will go wrong there?
Now also the block sizes for BucketCache has to be tuned considering the encoding factor.
 Ya it makes the size restrictions bit more clear.  So when there is compression also in place,
need to adjust the factor accordingly say 0.5 to 0.3 (eg:)

> Unify blocksize after encoding to decrease memory fragment 
> -----------------------------------------------------------
>
>                 Key: HBASE-17757
>                 URL: https://issues.apache.org/jira/browse/HBASE-17757
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Allan Yang
>            Assignee: Allan Yang
>         Attachments: HBASE-17757.patch
>
>
> Usually, we store encoded block(uncompressed) in blockcache/bucketCache. Though we have
set the blocksize, after encoding, blocksize is varied. Varied blocksize will cause memory
fragment problem, which will result in more FGC finally.In order to relief the memory fragment,
This issue adjusts the encoded block to a unified size.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message