hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3160) Compactions: Use more intelligent priorities for PriorityCompactionQueue
Date Wed, 03 Nov 2010 19:42:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927969#action_12927969
] 

Nicolas Spiegelberg commented on HBASE-3160:
--------------------------------------------

@Jeff: I forgot yesterday, but I also did fix an issue with the compaction priorities.  If
a flush happened for the store that was being compacted, it would be added to the compaction
queue at the pre-compact priority instead of post-compact.  We now do:
{code}
              if (!this.server.isStopRequested()) {
                // requests that were added during compaction will have a 
                // stale priority. remove and re-insert to update priority
                boolean hadCompaction = compactionQueue.remove(r);
                if (midKey != null) {
                  split(r, midKey);
                } else if (hadCompaction) {
                  compactionQueue.add(r);
                }
              }
{code}



> Compactions: Use more intelligent priorities for PriorityCompactionQueue
> ------------------------------------------------------------------------
>
>                 Key: HBASE-3160
>                 URL: https://issues.apache.org/jira/browse/HBASE-3160
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.89.20100924, 0.90.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-3160.patch
>
>
> One of the problems with the current compaction queue is that we have a very low granularity
on the importance of the various compactions in the queue.  If a StoreFile count exceeds 15
files, only then do we bump via enum change.  We should instead look into more intelligent,
granular priority metrics for choosing the next compaction.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message