Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B5D4FF65B for ; Wed, 3 Apr 2013 18:50:14 +0000 (UTC) Received: (qmail 20041 invoked by uid 500); 3 Apr 2013 18:50:09 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 19957 invoked by uid 500); 3 Apr 2013 18:50:09 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 19949 invoked by uid 99); 3 Apr 2013 18:50:09 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Apr 2013 18:50:09 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_FONT_SIZE_LARGE,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dwivedishashwat@gmail.com designates 209.85.217.174 as permitted sender) Received: from [209.85.217.174] (HELO mail-lb0-f174.google.com) (209.85.217.174) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Apr 2013 18:50:03 +0000 Received: by mail-lb0-f174.google.com with SMTP id s10so1933780lbi.19 for ; Wed, 03 Apr 2013 11:49:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=NLFx1vYyzabg29aWWdjB2mNQMtdVMoJV0PUThfHJ0aw=; b=B8jA6czE88sg4NMkC6kma8EekvX9C9E6CTCOh9zSmaumdLuFjRnCMCa7ubRAa6VMYm uF0eeHvKTd5tKFMbYx0je7t5rJv4PIpJFDtTLkvBfXrQ3h8Rb6qpzY5H4nMcB25DWqPK XsaK97VADi3eZFUXDrPSunxGB9zuYbJFkMTq0PpCFHH9SHKlF7IrmvvwucJgMLiLGJI5 Wft+bh4Vfnz5kVFHmLKzHbhEoPaw/RkiwRNCACC8qAy/DrDrLZpI59EqCFgaKKTo1Tav sJa2JcKNs4EjtZg9zZGCNuqNm6k4eTOR2o6OtijbaPnJks8V1v8DPsk/HylnXyQUb/rh gGwA== X-Received: by 10.112.29.72 with SMTP id i8mr1684458lbh.61.1365014983142; Wed, 03 Apr 2013 11:49:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.82.197 with HTTP; Wed, 3 Apr 2013 11:49:23 -0700 (PDT) In-Reply-To: References: From: shashwat shriparv Date: Thu, 4 Apr 2013 00:19:23 +0530 Message-ID: Subject: Re: NameNode failure and recovery! To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=f46d040169972efa4d04d97951bf X-Virus-Checked: Checked by ClamAV on apache.org --f46d040169972efa4d04d97951bf Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable If you are not in position to go for HA just keep your checkpoint period shorter to have recent data recoverable from SNN. and you always have a option hadoop namenode -recover try this on testing cluster and get versed to it. and take backup of image at some solid state storage. =E2=88=9E Shashwat Shriparv On Wed, Apr 3, 2013 at 9:56 PM, Harsh J wrote: > There is a 3rd, most excellent way: Use HDFS's own HA, see > > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHi= ghAvailabilityWithQJM.html > :) > > On Wed, Apr 3, 2013 at 8:10 PM, Rahul Bhattacharjee > wrote: > > Hi all, > > > > I was reading about Hadoop and got to know that there are two ways to > > protect against the name node failures. > > > > 1) To write to a nfs mount along with the usual local disk. > > -or- > > 2) Use secondary name node. In case of failure of NN , the SNN can take > in > > charge. > > > > My questions :- > > > > 1) SNN is always lagging , so when SNN becomes primary in event of a NN > > failure , then the edits which have not been merged into the image fil= e > > would be lost , so the system of SNN would not be consistent with the N= N > > before its failure. > > > > 2) Also I have read that other purpose of SNN is to periodically merge > the > > edit logs with the image file. In case a setup goes with option #1 > (writing > > to NFS, no SNN) , then who does this merging. > > > > Thanks, > > Rahul > > > > > > > > -- > Harsh J > --f46d040169972efa4d04d97951bf Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If you are not in position to go for HA jus= t keep your checkpoint period shorter to have recent data recoverable from = SNN.

and you always have a option
hadoop namenode -re= cover
try this on testing cluster and get versed to it.
and take backup of image at some solid state storage.

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 =09 =09 =09 =09

=E2=88=9E

Shashwat Shriparv



On Wed, Apr 3, 2013 at 9:56 PM, Harsh J = <harsh@cloudera.com> wrote:
There is a 3rd, most excellent way: Use HDFS's own HA, see
http://hadoop.apache= .org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithQJM.= html
:)

On Wed, Apr 3, 2013 at 8:10 PM, Rahul Bhattacharjee
<rahul.rec.dgp@gmail.com&= gt; wrote:
> Hi all,
>
> I was reading about Hadoop and got to know that there are two ways to<= br> > protect against the name node failures.
>
> 1) To write to a nfs mount along with the usual local disk.
> =C2=A0-or-
> 2) Use secondary name node. In case of failure of NN , the SNN can tak= e in
> charge.
>
> My questions :-
>
> 1) SNN is always lagging , so when SNN becomes primary in event of a N= N
> failure , =C2=A0then the edits which have not been merged into the ima= ge file
> would be lost , so the system of SNN would not be consistent with the = NN
> before its failure.
>
> 2) Also I have read that other purpose of SNN is to periodically merge= the
> edit logs with the image file. In case a setup goes with option #1 (wr= iting
> to NFS, no SNN) , then who does this merging.
>
> Thanks,
> Rahul
>
>



--
Harsh J

--f46d040169972efa4d04d97951bf--