hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5332) Deterministic Compaction Jitter
Date Tue, 21 Feb 2012 20:22:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212876#comment-13212876
] 

Nicolas Spiegelberg commented on HBASE-5332:
--------------------------------------------

@Prakash: are you thinking some sort of priority queue where we'll issue a major compaction
if the queue is empty and a minimum time has elapsed?  That's a possibility but would require
a stateful class to handle the problem.  We could just reject a compaction request if the
queue is too full to avoid state.  

However, the main issue is that the non-persistence causes large waves of major compactions
versus a small batch of 3 or 4.  The random jitter isn't really an issue because the compactions
should be put in the large compaction queue and not affect normal operation.  Really, this
is more of an inconvenience right now, so I think a small, non-disruptive fix is best.  Other
thoughts?
                
> Deterministic Compaction Jitter
> -------------------------------
>
>                 Key: HBASE-5332
>                 URL: https://issues.apache.org/jira/browse/HBASE-5332
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>         Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch
>
>
> Currently, we add jitter to a compaction using "delay + jitter*(1 - 2*Math.random())".
 Since this is non-deterministic, we can get major compaction storms on server restart as
half the Stores that were set to "delay + jitter" will now be set to "delay - jitter".  We
need a more deterministic way to jitter major compactions so this information can persist
across server restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message