hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3842) Refactor Coprocessor Compaction API
Date Thu, 01 Sep 2011 04:23:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095109#comment-13095109
] 

jiraposter@reviews.apache.org commented on HBASE-3842:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1686/#review1713
-----------------------------------------------------------

Ship it!



src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java
<https://reviews.apache.org/r/1686/#comment3887>

    Should this be in its own change set?



src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
<https://reviews.apache.org/r/1686/#comment3888>

    Nice test.


- Andrew


On 2011-09-01 02:35:36, Gary Helmling wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1686/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-09-01 02:35:36)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  This patch adds two new hooks to wrap the selection process for store files to compact:
bq.  
bq.    void preCompactSelection(final ObserverContext<RegionCoprocessorEnvironment>
c,
bq.        final Store store, final List<StoreFile> candidates);
bq.  
bq.    void postCompactSelection(final ObserverContext<RegionCoprocessorEnvironment>
c,
bq.        final Store store, final ImmutableList<StoreFile> selected);
bq.  
bq.  In addition, the existing preCompact and postCompact methods have been refactored as
described in JIRA:
bq.  
bq.    InternalScanner preCompact(final ObserverContext<RegionCoprocessorEnvironment>
c,
bq.      final Store store, final InternalScanner scanner);
bq.  
bq.    void postCompact(final ObserverContext<RegionCoprocessorEnvironment> c,
bq.      final Store store, StoreFile resultFile);
bq.  
bq.  Coprocessors that wish to override compaction behavior can wrap the provided InternalScanner
in preCompact and return their own implementation.  They can then apply custom policy on the
fly before returning KeyValues from the scanner.  Alternately, the coprocessor could set the
"bypass" flag in preCompact, which will skip the normal process of writing out a new store
file.  In this case, the coprocessor is indicating that it will handle the store file writing
itself.  Once the coprocessor has written a new store file, it will need to explicitly tell
the Store instance to load it using bulkLoadHFile().
bq.  
bq.  
bq.  This addresses bug HBASE-3842.
bq.      https://issues.apache.org/jira/browse/HBASE-3842
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java d473ba7 
bq.    src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java 008d027 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 30c9d69 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java 53645ce

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 655db7d 
bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java c0b7267

bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java c2af6a1 
bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java a8edb42

bq.    src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
b370ff0 
bq.  
bq.  Diff: https://reviews.apache.org/r/1686/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Added a new test case -- TestRegionObserverInterface#testCompactionOverride -- to verify
custom compaction handling using an InternalScanner implementation.
bq.  
bq.  All additional coprocessor related tests pass.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Gary
bq.  
bq.



> Refactor Coprocessor Compaction API
> -----------------------------------
>
>                 Key: HBASE-3842
>                 URL: https://issues.apache.org/jira/browse/HBASE-3842
>             Project: HBase
>          Issue Type: Improvement
>          Components: coprocessors, regionserver
>    Affects Versions: 0.92.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Gary Helmling
>            Priority: Critical
>              Labels: compaction
>             Fix For: 0.92.0
>
>
> After HBASE-3797, the compaction logic flow has been significantly altered.  Because
of this, the current compaction coprocessor API is insufficient for gaining full insight into
compaction requests/results.  Refactor coprocessor API after HBASE-3797 is committed to be
more extensible and increase visibility.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message