hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14326) HBase book: fix definition of max min size to compact
Date Thu, 22 Oct 2015 08:45:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14968804#comment-14968804

Hudson commented on HBASE-14326:

FAILURE: Integrated in HBase-TRUNK #6937 (See [https://builds.apache.org/job/HBase-TRUNK/6937/])
HBASE-14326 Clarify that ExploringCompactionPolicy evaluates a selection (mstanleyjones: rev
* hbase-common/src/main/resources/hbase-default.xml

> HBase book: fix definition of max min size to compact
> -----------------------------------------------------
>                 Key: HBASE-14326
>                 URL: https://issues.apache.org/jira/browse/HBASE-14326
>             Project: HBase
>          Issue Type: Bug
>          Components: documentation
>            Reporter: Vladimir Rodionov
>            Assignee: Misty Stanley-Jones
>             Fix For: 2.0.0
>         Attachments: HBASE-14326.patch
> I think we need to change wording/definition of these config parameters in HBase book,
they are misleading:
> {quote}
> hbase.hstore.compaction.min.size
> Description
> A StoreFile smaller than this size will always be eligible for minor compaction. HFiles
this size or larger are evaluated by hbase.hstore.compaction.ratio to determine if they are
eligible. Because this limit represents the "automatic include"limit for all StoreFiles smaller
than this value, this value may need to be reduced in write-heavy environments where many
StoreFiles in the 1-2 MB range are being flushed, because every StoreFile will be targeted
for compaction and the resulting StoreFiles may still be under the minimum size and require
further compaction. If this parameter is lowered, the ratio check is triggered more quickly.
This addressed some issues seen in earlier versions of HBase but changing this parameter is
no longer necessary in most situations. Default: 128 MB expressed in bytes.
> Default
> 134217728
> hbase.hstore.compaction.max.size
> Description
> A StoreFile larger than this size will be excluded from compaction. The effect of raising
hbase.hstore.compaction.max.size is fewer, larger StoreFiles that do not get compacted often.
If you feel that compaction is happening too often without much benefit, you can try raising
this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.
> hbase.hstore.compaction.ratio
> Description
> For minor compaction, this ratio is used to determine whether a given StoreFile which
is larger than hbase.hstore.compaction.min.size is eligible for compaction. Its effect is
to limit compaction of large StoreFiles. The value of hbase.hstore.compaction.ratio is expressed
as a floating-point decimal. A large ratio, such as 10, will produce a single giant StoreFile.
Conversely, a low value, such as .25, will produce behavior similar to the BigTable compaction
algorithm, producing four StoreFiles. A moderate value of between 1.0 and 1.4 is recommended.
When tuning this value, you are balancing write costs with read costs. Raising the value (to
something like 1.4) will have more write costs, because you will compact larger StoreFiles.
However, during reads, HBase will need to seek through fewer StoreFiles to accomplish the
read. Consider this approach if you cannot take advantage of Bloom filters. Otherwise, you
can lower this value to something like 1.0 to reduce the background cost of writes, and use
Bloom filters to control the number of StoreFiles touched during reads. For most cases, the
default value is appropriate.
> Default
> 1.2F
> {quote}
> For details, see HBASE-14263.

This message was sent by Atlassian JIRA

View raw message