hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon" <t...@cloudera.com>
Subject Re: Review Request: Cache flush should go to tmp dir
Date Tue, 22 Jun 2010 22:55:16 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/216/
-----------------------------------------------------------

(Updated 2010-06-22 15:55:16.514925)


Review request for hbase, stack and Ryan Rawson.


Changes
-------

TestMasterTransitions exposed a clunker of a bug here. In the previous iteration, I had put
the tmp directory in /hbase/table/.tmp instead of /hbase/table/region/.tmp - I thought region.basedir
was the region dir, not the table dir. So, when a new region was opened, it removed the tmp
files being written by other region servers and made the world explode.

This iteration renames those variables to be more clear, and fixes the bug so the tmp dir
is within the region, not the table.


Summary
-------

Fixes bugs where an exception in the middle of flushing a file leaves a half-written StoreFile
in the region dir, preventing that region from recovering, or, in the case of transient errors,
causing silent loss of half a file's worth of data.

I also got rid of the compaction dir here, and am just using one region-wide tmp dir. Is there
some reason this is a bad idea?


This addresses bug HBASE-2729.
    http://issues.apache.org/jira/browse/HBASE-2729


Diffs (updated)
-----

  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1794df8 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 04b7522 
  src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java dc38b3b 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java 9e5ca46 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java a65e947 

Diff: http://review.hbase.org/r/216/diff


Testing
-------

Ran TestCompaction and TestStore. Will start a cluster test running before I go to bed.


Thanks,

Todd


Mime
View raw message