hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From st...@duboce.net
Subject Re: Review Request: HBASE-2461 Split doesn't handle IOExceptions when creating new region reference files
Date Tue, 03 Aug 2010 20:10:44 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2010-08-03 13:10:44.610359)

Review request for hbase.


Found minor issue where wasnt' returning if failed rollback.


Posting this to review board.

Patch that keeps a journal during split transaction.  If split fails, call to rollback will
restore the parent to original open condition by backing up whatever transaction steps completed.

The transaction spans split checks, closing of parent region and creation of daughters up
to the addition of parent offlining to .META.  Once the .META. edit has been made, we cannot
rollback -- we have to go forward.  This means that the basescanner fixup that will add missing
daughter regions should the regionserver crash after parent region edit but before its added
daughters is still required, in some form at least.

This patch includes a test of the new split code but only run against an HRegion, not in server
context.  The split code is buried in heart of the regionserver and created on startup.  I
stared at it for a while and injecting fault was just forbidding.  Its like bramble; there
are so many spikes in the way of getting your finger down into the running split I ended up
passing on it.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
(split): Most of the split code has been moved out to the new SplitTransaction class.
Now this method prepares the split transaction, executes, and if failure does rollback.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(splitLock) Removed. Doesn't seem necessary.  Just  made close method synchronized.
(SPLITDIR) Moved to new SplitTransaction
Moved cleanup of half-done splits into SplitTransaction.  It'll know better how to do this.
Moved split code into SplitTransaction class.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Made this class implement new OnlineRegions interface

+ A src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
New Interface that allows you add/remove regions from oline regions.  This Interface
adds little.  Was just trying to make it so I didn't have to have server context doing
tests but in the end I just passed null for the case of no server context.  Could remove

+ A src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
New class that encapsulates all to do w/ splitting "transaction".

+ A src/main/java/org/apache/hadoop/hbase/util/PairOfSameType.java
Minor utility class

+M src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
(loadRegion) Added loading a region

+ M src/test/java/org/apache/hadoop/hbase/io/TestImmutableBytesWritable.java
(testSpecificCompare) Unrelated change

+ M src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Change because of new manner in which splits are run.  Added a splitRegions method.

+ A src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
Test of region splitting code in region context.  Testing in server context would take
a bunch of work making it so could insert mock instance of SplitTransaction.

This addresses bug HBASE-2461.

Diffs (updated)

  src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 7589db3 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 6dc41a4 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 6a54736 
  src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/util/PairOfSameType.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 4d09fe9 
  src/test/java/org/apache/hadoop/hbase/io/TestImmutableBytesWritable.java 43fa6dd 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 98bd3e5 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java PRE-CREATION

Diff: http://review.cloudera.org/r/474/diff


Basic unit tests are passing.



View raw message