hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5867) Improve Compaction Throttle Default
Date Fri, 04 May 2012 15:38:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268458#comment-13268458

Nicolas Spiegelberg commented on HBASE-5867:

@stack: this should be a much better out-of-the-box than we currently have.  If anything,
we could make the value higher (4 or 8 instead of 2).  However, this should be a substantially-better
default than the current.
> Improve Compaction Throttle Default
> -----------------------------------
>                 Key: HBASE-5867
>                 URL: https://issues.apache.org/jira/browse/HBASE-5867
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>         Attachments: D2943.1.patch
> We recently had a production issue where our compactions fell behind because our compaction
throttle was improperly tuned and accidentally upgraded all compactions to the large pool.
 The default from HBASE-3877 makes 1 bad assumption: the default number of flushed files in
a compaction.  Currently the algorithm is:
> throttleSize ~= flushSize * 2
> This assumes that the basic compaction utilizes 3 files and that all 3 files are compressed.
 In this case, "hbase.hstore.compaction.min" == 6 && the values were not very compressible.
 Both conditions should be taken into consideration.  As a default, it is less damaging for
the large thread to be slightly higher than it needs to be versus having everything accidentally

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message