hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5332) Deterministic Compaction Jitter
Date Mon, 20 Feb 2012 02:31:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211641#comment-13211641
] 

Phabricator commented on HBASE-5332:
------------------------------------

nspiegelberg has commented on the revision "[jira] [HBASE-5332] Deterministic Compaction Jitter".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1067 I can null check.  note
that this variable is not null-checked in a lot of places in Store.java.  Wish there was a
way to guarantee non-null
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1065 I was worried about a
case where this check happened while a major compaction was finishing up  (see Store.completeCompaction).
 Besides per-file compactions, the HRegionServer.CompactionChecker thread is another area
where this race can occur.

  Maybe this isn't a practical problem since we don't cache this calculation anymore &
the major compaction interval should normally be very large so a race condition won't trigger
a major compaction.
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:1068 Thinking about it more,
the pathname alone should suffice since it's a 'crypographically strong' random UUID

REVISION DETAIL
  https://reviews.facebook.net/D1785

                
> Deterministic Compaction Jitter
> -------------------------------
>
>                 Key: HBASE-5332
>                 URL: https://issues.apache.org/jira/browse/HBASE-5332
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>         Attachments: D1785.1.patch, D1785.2.patch
>
>
> Currently, we add jitter to a compaction using "delay + jitter*(1 - 2*Math.random())".
 Since this is non-deterministic, we can get major compaction storms on server restart as
half the Stores that were set to "delay + jitter" will now be set to "delay - jitter".  We
need a more deterministic way to jitter major compactions so this information can persist
across server restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message