hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3160) Compactions: Use more intelligent priorities for PriorityCompactionQueue
Date Wed, 27 Oct 2010 18:49:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925502#action_12925502
] 

Nicolas Spiegelberg commented on HBASE-3160:
--------------------------------------------

My initial suggestion:

Priority = max(len(s.storefiles) for s in region.stores)

This will preserve the priority of Stores exceeding the 15 file limit while prioritizing Stores
that are getting close to exceeding that limit.  As a further topic of discussion, it would
be good to interrupt a low-pri compaction (< 15 pri) when a 15+ pri request comes in. 
Of course, a lot of this would be nicer if we had a compaction thread pool. A man can dream...

> Compactions: Use more intelligent priorities for PriorityCompactionQueue
> ------------------------------------------------------------------------
>
>                 Key: HBASE-3160
>                 URL: https://issues.apache.org/jira/browse/HBASE-3160
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>
> One of the problems with the current compaction queue is that we have a very low granularity
on the importance of the various compactions in the queue.  If a StoreFile count exceeds 15
files, only then do we bump via enum change.  We should instead look into more intelligent,
granular priority metrics for choosing the next compaction.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message