hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2462) Review compaction heuristic and move compaction code out so standalone and independently testable
Date Mon, 25 Oct 2010 03:58:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924442#action_12924442

HBase Review Board commented on HBASE-2462:

Message from: "Nicolas" <nspiegelberg@facebook.com>

This is an automatically generated e-mail. To reply, visit:


    If HBASE-3102 goes in first, this section just needs to add:
     if (r.getLastCompactInfo() != null) {  // compaction aborted?


    need to recompute totalsize here to substract the filesizes of the skipped storefiles.
 this is necessary for the metrics collection, in addition to this debug


    1. it would be nice if the Major vs. Minor compaction distinction was handled by the CompactionSelector
itself.  The constructor could take a CompactionRequestor class to handle this.
    2. it would be nice to add shouldCompact() to the CompactionSelector so that custom compaction
algorithms can determine when they are queued.


    I think it's a great idea to start some sort of benchmarking utility for this.  I know
this is a first cut, but it would be nice to have a quick intro guide or something in here
for hooking your custom compaction algo into this benchmark.  Comments are a little sparse
here about what's going on.

- Nicolas

> Review compaction heuristic and move compaction code out so standalone and independently
> -------------------------------------------------------------------------------------------------
>                 Key: HBASE-2462
>                 URL: https://issues.apache.org/jira/browse/HBASE-2462
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>            Assignee: Jonathan Gray
>            Priority: Critical
> Anything that improves our i/o profile makes hbase run smoother.  Over in HBASE-2457,
good work has been done already describing the tension between minimizing compactions versus
minimizing count of store files.  This issue is about following on from what has been done
in 2457 but also, breaking the hard-to-read compaction code out of Store.java out to a standalone
class that can be the easier tested (and easily analyzed for its performance characteristics).
> If possible, in the refactor, we'd allow specification of alternate merge sort implementations.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message