hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5332) Deterministic Compaction Jitter
Date Sat, 04 Feb 2012 01:09:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200233#comment-13200233
] 

Nicolas Spiegelberg commented on HBASE-5332:
--------------------------------------------

The current proposal is to seed the random number generator with existing persistent data:
{code}
 java.util.Random(hash(Region Name + oldest file timestamp)).nextDouble()  
{code}

Note that that the currently algorithm is normally an annoyance in 92 and above because major
compactions will normally go in the large compaction queue.  However, 90 and users who have
the large compaction queue incorrectly configured could experience high latency after restart
because a lot of major compactions will pile up.
                
> Deterministic Compaction Jitter
> -------------------------------
>
>                 Key: HBASE-5332
>                 URL: https://issues.apache.org/jira/browse/HBASE-5332
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>
> Currently, we add jitter to a compaction using "delay + jitter*(1 - 2*Math.random())".
 Since this is non-deterministic, we can get major compaction storms on server restart as
half the Stores that were set to "delay + jitter" will now be set to "delay - jitter".  We
need a more deterministic way to jitter major compactions so this information can persist
across server restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message