hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3927) display total uncompressed byte size of a region in web UI
Date Tue, 31 May 2011 16:04:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041643#comment-13041643

Ted Yu commented on HBASE-3927:

Thanks for the hint, Stack.
Looking at RegionLoad.readFields(), I am a little confused by the handling of RegionLoad.VERSION.
getVersion() just returns the static VERSION, making the subsequent version check almost meaningless.
I plan to increase RegionLoad.VERSION and persist the version byte for TRUNK.

> display total uncompressed byte size of a region in web UI
> ----------------------------------------------------------
>                 Key: HBASE-3927
>                 URL: https://issues.apache.org/jira/browse/HBASE-3927
>             Project: HBase
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Matt Corgan
>            Assignee: Ted Yu
>            Priority: Minor
>         Attachments: 3927.txt, regionserver-showing-compression-ratio.png
> The decision to split data blocks when flushing and compacting is made based on the uncompressed
data size which can often lead to compressed disk blocks that are a fraction of the intended
64 KB (default).  This often leads to a larger number of blocks and index entries than expected
and can cause block indexes to take up GB of memory.
> There is already a "long totalUncompressedBytes" written to the HFileTrailer.  It would
be nice to expose this in the web UI to make it easier to calculate the compression ratio
and then raise the block size appropriately (not necessarily to get it back to 64K).
> This should probably be added wherever the other HFile metrics are: RegionLoad.createRegions(..),
and HServerLoad.  HServerLoad is a Writable, so it may break serialization.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message