hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3797) StoreFile Level Compaction Locking
Date Wed, 04 May 2011 17:04:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028824#comment-13028824
] 

jiraposter@reviews.apache.org commented on HBASE-3797:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/693/#review646
-----------------------------------------------------------


This looks great.  Some comments in below.  Have you tried it Nicolas?  Does it work for you?
 I'm game for getting this into TRUNK sooner rather than earlier but was thinking the default
settings conservative.  Should we have at least one small compactions thread running (Default
is zero).


src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
<https://reviews.apache.org/r/693/#comment1293>

    Funny. What you doing at walmart?  Oh yeah, you are from the south!



src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
<https://reviews.apache.org/r/693/#comment1294>

    Deprecate this in favor of the new API?  Or just remove this one since its an internal-only
API?



src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
<https://reviews.apache.org/r/693/#comment1304>

    How we ensure a compaction and split of same region do not clash?  Or is that not a prob.
in this redo?



src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
<https://reviews.apache.org/r/693/#comment1300>

    Missing license



src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
<https://reviews.apache.org/r/693/#comment1301>

    Is this good name for this?  Would 'Split' be a better name?  Or 'Splitter'



src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
<https://reviews.apache.org/r/693/#comment1302>

    Can you pass this on construction?  Seems like something that should not be changed post-creation



src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
<https://reviews.apache.org/r/693/#comment1303>

    This is a nice refactoring.



src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
<https://reviews.apache.org/r/693/#comment1299>

    Did we remove this.  Is it not needed any more?



src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
<https://reviews.apache.org/r/693/#comment1296>

    The class comment is off now, is that right?  Now it does more than hold the details.
 Now it actually runs the compaction? If so, should we rename the class?



src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
<https://reviews.apache.org/r/693/#comment1297>

    You have white space throughout your patch.  No biggie but you might want to fix on commit.



src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
<https://reviews.apache.org/r/693/#comment1298>

    Do we check that the files that make up the compaction are still around when we go to
run?  Is it possible they might have changed between queu'ing and running?



src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
<https://reviews.apache.org/r/693/#comment1295>

    Good


- Michael


On 2011-05-04 15:48:58, Michael Stack wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/693/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-05-04 15:48:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  hbase-3797 StoreFile Level Compaction Locking
bq.  
bq.  I posted this here for nicolas
bq.  
bq.  
bq.  This addresses bug hbase-3797.
bq.      https://issues.apache.org/jira/browse/hbase-3797
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 24450ae

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java fdbf659

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 11fd50e 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java b910254 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java f66a7cd 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/PriorityCompactionQueue.java 13bcb3f

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/Store.java e2295c2 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
98f962b 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
345e393 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java 6517ba7

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java
c5d876d 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java adfe1f8 
bq.  
bq.  Diff: https://reviews.apache.org/r/693/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Michael
bq.  
bq.



> StoreFile Level Compaction Locking
> ----------------------------------
>
>                 Key: HBASE-3797
>                 URL: https://issues.apache.org/jira/browse/HBASE-3797
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>         Attachments: HBASE-3797+1476.patch
>
>
> Multithreaded compactions (HBASE-1476) will solve the problem of major compactions clogging
high-priority minor compactions.  However, there is still a problem here.  Since compactions
are store-level, the store undergoing major compaction will have it's storefile count increase
during the major.  We really need a way to allow multiple outstanding compactions per store.
 compactSelection() should lock/reserve the files being used for compaction.  This will also
allow us to know what we're going to compact when inserting into the CompactSplitThread and
make more informed priority queueing decisions.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message