hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lei liu <liulei...@gmail.com>
Subject Re: same edits file is loaded more than once
Date Fri, 09 Nov 2012 06:33:34 GMT
Hi Coli,
Thank you very much for your reply.

I am researching the hadoop2.0 code, I find there are below code in
FSEditLogLoader.loadEditRecords method.
<code>
   if (op.hasTransactionId()) {
            if (op.getTransactionId() > expectedTxId) {
              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
                  "to be a gap in the edit log.  We expected txid " +
                  expectedTxId + ", but got txid " +
                  op.getTransactionId() + ".", recovery, "ignoring missing
" +
                  " transaction IDs");
            } *else if (op.getTransactionId() < expectedTxId) {
              MetaRecoveryContext.**editLogLoaderPrompt("There appears " +
                  "to be an out-of-order edit in the edit log.  We " +
                  "expected txid " + expectedTxId + ", but got txid " +
                  op.getTransactionId() + ".", recovery,
                  "skipping the out-of-order edit");
              continue;*
            }
          }
</code>

The method use transaction id to  **guarantee same transaction log is not
applied to namespace more than once.

I am using hadoop-0.20.2, that don't store the transaction id to edits log
file.
So I want to know if  StandbyNN apply same edits log file many times to
namespace, that will lead to the namespace of StandbyNN is corrupt?






2012/11/8 Colin McCabe <cmccabe@alumni.cmu.edu>

> Hi,
>
> If you want to learn more about HA in HDFS, here are some slides from
> a talk that Aaron T. Meyers and Suresh Srinivas gave:
> http://www.slideshare.net/hortonworks/nn-ha-hadoop-worldfinal-10173419
>
> branch-2 and later contain HDFS HA.
>
> cheers,
> Colin
>
>
> On Sun, Nov 4, 2012 at 1:06 AM, lei liu <liulei412@gmail.com> wrote:
> > I am using hadoop0.20.2, now I want to use HDFS HA function.  I research
> > AvatarNode. I find if the StandbyNN do checkpoint fail, when next time
> the
> > StandbyNN do checkpoint, the same edits file is loaded again.  Can same
> > edits file be loaded more than once in hadoop0.20.2?
> > if not, what is the harm?
> >
> > Thanks,
> >
> > LiuLei
> >
>

Mime
View raw message