hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5332) Deterministic Compaction Jitter
Date Mon, 20 Feb 2012 02:31:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211641#comment-13211641

Phabricator commented on HBASE-5332:

nspiegelberg has commented on the revision "[jira] [HBASE-5332] Deterministic Compaction Jitter".

  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1067 I can null check.  note
that this variable is not null-checked in a lot of places in Store.java.  Wish there was a
way to guarantee non-null
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1065 I was worried about a
case where this check happened while a major compaction was finishing up  (see Store.completeCompaction).
 Besides per-file compactions, the HRegionServer.CompactionChecker thread is another area
where this race can occur.

  Maybe this isn't a practical problem since we don't cache this calculation anymore &
the major compaction interval should normally be very large so a race condition won't trigger
a major compaction.
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1068 Thinking about it more,
the pathname alone should suffice since it's a 'crypographically strong' random UUID


> Deterministic Compaction Jitter
> -------------------------------
>                 Key: HBASE-5332
>                 URL: https://issues.apache.org/jira/browse/HBASE-5332
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>         Attachments: D1785.1.patch, D1785.2.patch
> Currently, we add jitter to a compaction using "delay + jitter*(1 - 2*Math.random())".
 Since this is non-deterministic, we can get major compaction storms on server restart as
half the Stores that were set to "delay + jitter" will now be set to "delay - jitter".  We
need a more deterministic way to jitter major compactions so this information can persist
across server restarts.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message