hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5616) Make compaction code standalone
Date Fri, 23 Mar 2012 17:03:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236788#comment-13236788
] 

jiraposter@reviews.apache.org commented on HBASE-5616:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4469/
-----------------------------------------------------------

Review request for hbase.


Summary
-------

Introduces a standalone CompactionTool under src/test. Can call it from its main
and have it run compactions on arbitrary storefiles.

Compaction code is moved out of Store into a new Compactor class.

CompactionTool needs a Store and a mocked up Region to run because
compacting uses a StoreScanner (A StoreScanner needs a Store. A
Store needs an HRegion). Rather than expect to be passed a coherent
HRegion pointer, instead, we fake up one using Mockito.

I tried to break out of HRegion a "Region" Interface. This Region Interface
would have a basic subset of HRegion functionality and we'd pass this
instead of HRegion to SplitTransaction, Store, CompactionRequest, etc.
but the change would be massive. Everything expects to be able to do
anything on an HRegion. This is work we need to do but I"m not doing
it as part of this patch.

M src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
Use new CompactionTool instead of asking Store directly to compact.
M src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java
Formatting.
M src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
Format the log message so properly spaced.
A src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
New compactiontool. Runs all compactions.
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Make getOpenAndCloseThreadPool static.
M src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
Move compaction code out to CompactionTool.
Refactor so can override ttl and Store home directory so Store
is more mockable; can now stand up a Store on its own w/o real
HRegion context.
M src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
Formatting.


This addresses bug hbase-5616.
    https://issues.apache.org/jira/browse/hbase-5616


Diffs
-----

  src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java b52e5d3 
  src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 4efdc6b 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java dcede5a 
  src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionProgress.java 553eee0

  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java d2329e1 
  src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 4bfd42f 
  src/test/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java 91ac652 

Diff: https://reviews.apache.org/r/4469/diff


Testing
-------


Thanks,

Michael


                
> Make compaction code standalone
> -------------------------------
>
>                 Key: HBASE-5616
>                 URL: https://issues.apache.org/jira/browse/HBASE-5616
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>            Assignee: stack
>         Attachments: 5616.txt, 5616v3.txt, 5616v6.txt, standalone.txt
>
>
> This is part of hbase-2462.  Make the compaction code standalone so can run it independent
of hbase.  Will make it easier to profile and try stuff out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message