hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon" <t...@cloudera.com>
Subject Re: Review Request: HBASE-2312: Rename HLog Dir when Splitting
Date Wed, 28 Jul 2010 17:17:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/396/#review513
-----------------------------------------------------------


This looks good, except that it requires the HDFS patches to be in place to even compile.
Although I think it's OK to not fix the bug in the case that we're on stock HDFS, I think
we still need to be able to run, buggily.

Did we determine that all of the other solutions were flawed/too complicated?


trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<http://review.cloudera.org/r/396/#comment2061>

    Check return value of rename (this is one of those stupid APIs that returns false instead
of throwing)



trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<http://review.cloudera.org/r/396/#comment2062>

    rather than reassigning, just pass splitDir below?



trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java
<http://review.cloudera.org/r/396/#comment2064>

    extract "-splitting" out to a constant?
    Or can we move some of this common code into HLog?



trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java
<http://review.cloudera.org/r/396/#comment2063>

    maybe:
    assert !(logDirExists && splitDirExists) : "Both " + rsLogDir + " and " + rsSplitDir
+ " exist";
    so if the assertion failure happens it's more understandable


- Todd


On 2010-07-27 23:10:02, Nicolas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/396/
> -----------------------------------------------------------
> 
> (Updated 2010-07-27 23:10:02)
> 
> 
> Review request for hbase.
> 
> 
> Summary
> -------
> 
> There is a very corner case when bad things could happen(ie data loss):
> 
> 1) RS #1 is going to roll its HLog - not yet created the new one, old one will get no
more writes
> 2) RS #1 enters GC Pause of Death
> 3) Master lists HLog files of RS#1 that is has to split as RS#1 is dead, starts splitting
> 4) RS #1 wakes up, created the new HLog (previous one was rolled) and appends an edit
- which is lost
> 
> Note that this fix requires a healthy dose of HDFS prerequisites: HDFS-617, HADOOP-6840,
HADOOP-6886.  I encourage you to review those as well, give feedback, and hopefully give +1s
so we can push the changes through.
> 
> 
> This addresses bug HBASE-2312.
>     http://issues.apache.org/jira/browse/HBASE-2312
> 
> 
> Diffs
> -----
> 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 979953 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java 979953

>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 979953 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java
979953 
>   trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java 979953

> 
> Diff: http://review.cloudera.org/r/396/diff
> 
> 
> Testing
> -------
> 
> mvn test;
> bin/start-hbase.sh
> bin/hbase shell < scan '.META.', get, put, etc
> 
> 
> Thanks,
> 
> Nicolas
> 
>


Mime
View raw message