hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8299) ExploringCompactionPolicy can get stuck in rare cases.
Date Tue, 09 Apr 2013 20:18:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627041#comment-13627041

Elliott Clark commented on HBASE-8299:

bq.(I assume you mean no compaction ever selected?)
Nope.  I mean an infinite loop of compaction selection.  If the number of store files doesn't
change and you are over the high water mark then after a compaction is done another selection
will be queued.

So there have been problems in the real world with previous compaction selection algorithms
not choosing files to compact.  It is a bug if there are no files chosen when a compaction
is queued.  It's not as rare as you seem to think.  Ask Stack about the half size file problem.

This change only affects the algorith when it can't find any files sets in ratio and have
need of a compaction.  Just ignoring the need for a compaction is not an option.

bq.The intuitive reason to compact 10 1 100 (for example) from [ 10 1 100 5000 1 ] is relative
size to the 5000 file. Maybe that could be encoded in a heuristic.
That's a completely different algorithm.  If you want to write that then I would feel better
doing that in a different issue.  Right now we need to fix the policy that's currently there.

> ExploringCompactionPolicy can get stuck in rare cases.
> ------------------------------------------------------
>                 Key: HBASE-8299
>                 URL: https://issues.apache.org/jira/browse/HBASE-8299
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.95.1
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>         Attachments: HBASE-8299-0.patch
> If the files are very oddly sized then it's possible that ExploringCompactionPolicy can
get stuck.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message