hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3842) Refactor Coprocessor Compaction API
Date Tue, 02 Aug 2011 16:31:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13076296#comment-13076296

Andrew Purtell commented on HBASE-3842:

bq. are we expecting the common case to be loading a single "CompactionObserver" that overrides
the compaction implementation, or loading multiple that each override/customize compaction
policy but not necessarily behavior?

I believe in general Coprocessors will not usually be used to wholesale replace anything.
People have that option, but the design rationale is about supporting point changes and enhancements.
If we do it right, the interception points are designed so the CP implementer needs do only
the minimum work necessary to achieve their aims. So I think we will see extensions that in
this case here filter out or rewrite specific sets of KVs during compaction, as part of a
larger set of smallish changes to other functions that also do something different for specific
sets of KVs, or a particular key pattern meaningful to the CP application, etc. We need an
API that allows the CP implementer to remain concerned only with their application/extension,
not be required to reimplement compaction and understand all the hairy details if they want
to alter it even slightly.

bq. One alternative would be to have preCompact take the scanner to be used as a parameter,
as suggested, and return a scanner instance that would allow overriding policy and mutating
KVs, while still relying on the core writer functionality. 


But we must not have behind the scenes a stacked coprocessor configuration resulting in rewriting
store files over and over, if three stacked coprocessors running compaction three (or four!)
times over.

> Refactor Coprocessor Compaction API
> -----------------------------------
>                 Key: HBASE-3842
>                 URL: https://issues.apache.org/jira/browse/HBASE-3842
>             Project: HBase
>          Issue Type: Improvement
>          Components: coprocessors, regionserver
>    Affects Versions: 0.92.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>              Labels: compaction
>             Fix For: 0.92.0
> After HBASE-3797, the compaction logic flow has been significantly altered.  Because
of this, the current compaction coprocessor API is insufficient for gaining full insight into
compaction requests/results.  Refactor coprocessor API after HBASE-3797 is committed to be
more extensible and increase visibility.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message