hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ac@hsk.hk" ...@hsk.hk>
Subject Re: HA for hadoop-0.20.2
Date Tue, 13 Nov 2012 11:25:52 GMT
Hi, A question, is 2.x ready for production deployment?  Thanks

On 13 Nov 2012, at 5:19 PM, Harsh J wrote:

> Hi,
> 
> Why not just use the 2.x releases for HA-NNs? There is quite a wide
> delta between 0.20.x and 2.x, especially around the edit log areas
> after HDFS-1073.
> 
> In any case, I think your question suits hdfs-dev@hadoop.apache.org
> more than the user lists, although I don't quite understand what
> you're attempting to do (or point).
> 
> On Tue, Nov 13, 2012 at 2:18 PM, lei liu <liulei412@gmail.com> wrote:
>> I want to implement HA function for hadoop-0.20.2.
>> 
>> When I learn the hadoop-2.0 code, I meet some question like this:
>> 
>> Thera are below code in FSEditLogLoader.loadEditRecords method.
>> 
>>   if (op.hasTransactionId()) {
>>            if (op.getTransactionId() > expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be a gap in the edit log.  We expected txid " +
>>                  expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery, "ignoring missing "
>> +
>>                  " transaction IDs");
>>            } else if (op.getTransactionId() < expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be an out-of-order edit in the edit log.  We " +
>>                  "expected txid " + expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery,
>>                  "skipping the out-of-order edit");
>>              continue;
>>            }
>>          }
>> 
>> The method use transaction id to  guarantee same transaction log is not
>> applied to namespace more than once.
>> 
>> But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
>> log file. So I want to know if  StandbyNN apply same transaction log  to
>> namespace more than once, that will lead to the namespace of StandbyNN is
>> corrupt?
>> 
>> Please give me some adviceļ¼ŒThanks.
>> 
>> 
>> 
>> Best Regards
>> 
>> LiuLei
> 
> 
> 
> -- 
> Harsh J


Mime
View raw message