Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 03B89F7A8 for ; Sun, 7 Apr 2013 14:01:51 +0000 (UTC) Received: (qmail 491 invoked by uid 500); 7 Apr 2013 14:01:45 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 378 invoked by uid 500); 7 Apr 2013 14:01:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 370 invoked by uid 99); 7 Apr 2013 14:01:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 14:01:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of azuryyyu@gmail.com designates 209.85.223.177 as permitted sender) Received: from [209.85.223.177] (HELO mail-ie0-f177.google.com) (209.85.223.177) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 14:01:39 +0000 Received: by mail-ie0-f177.google.com with SMTP id tp5so5919734ieb.22 for ; Sun, 07 Apr 2013 07:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=8hL2m2nkMVpj9gMuXKklAwalWjLnoc6eLxaYzRBJmDI=; b=WX31zW8V81L5eqpVqBnQfOWw7QPn5qcig0UQCMOK0oM9Q7rmh0A0azafafw4O6dQNl e7IsgiVZQSEuDqsyEBhSYsFUu/uzHveZKFkO5tCDJFGdwjWxTpOUqxG9FndNke7lEt4v 1kp4k9Yk7/hNhvFeI/uLs13mFhHXC5i4ipCMNIaOMVtVfcV8GfKN5d7fMnH+bG0h/8Kw rlTTMydS1st3bTNFtENmSStjovgjONXgQDwcxJ/qWClxC4x5AKghE+QsgQcMax2gwyB6 L6VNxPYeU1HtbEcsjeluGg3jElsVwoGo5O68hWH+/110ZD2o/abVL5YP2dPDISIyvM5z /cqQ== MIME-Version: 1.0 X-Received: by 10.50.1.38 with SMTP id 6mr4152072igj.106.1365343278217; Sun, 07 Apr 2013 07:01:18 -0700 (PDT) Received: by 10.64.8.7 with HTTP; Sun, 7 Apr 2013 07:01:18 -0700 (PDT) Received: by 10.64.8.7 with HTTP; Sun, 7 Apr 2013 07:01:18 -0700 (PDT) In-Reply-To: References: Date: Sun, 7 Apr 2013 22:01:18 +0800 Message-ID: Subject: Re: backup node question From: Azuryy Yu To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e89a8f839fc318922804d9c5c1c0 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f839fc318922804d9c5c1c0 Content-Type: text/plain; charset=ISO-8859-1 I am confused. Hadoopv2 has NN SNN DN JN(journal node), so whats Standby Namenode? --Send from my Sony mobile. On Apr 7, 2013 9:03 PM, "Harsh J" wrote: > BackupNameNode is not present in the maintenance 1.x releases, it is a > feature added to a higher version; you can try it out in 2.x today if > you wish to. > > On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu wrote: > > Hi Harsh, > > Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x? > > > > > > On Sun, Apr 7, 2013 at 4:05 PM, Harsh J wrote: > >> > >> Yes, it need not keep an edits (transactions) stream locally cause > >> those are passed synchronously to the BackupNameNode, which persists > >> it on its behalf. > >> > >> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma wrote: > >> > Thanks Harsh, > >> > > >> > For your comments, "What it means is that the NameNode need not store > >> > anything locally", you mean Primary Name Node do not need to store > >> > checkpoint/journal locally, and only need to keep memory image > >> > up-to-date > >> > for edits? > >> > > >> > regards, > >> > Lin > >> > > >> > > >> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J wrote: > >> >> > >> >> Hi Lin, > >> >> > >> >> My reply inline. > >> >> > >> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma wrote: > >> >> > Hi guys, > >> >> > > >> >> > I am reading from this paper to learn about backup nodes > >> >> > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf), > >> >> > > >> >> > It is mentioned, "It contains all file system metadata information > >> >> > except > >> >> > for block locations. It can perform all operations of the regular > >> >> > NameNode > >> >> > that do not involve modification of the namespace or knowledge of > >> >> > block > >> >> > locations. ", what kinds of operations do not need knowledge of > block > >> >> > locations? > >> >> > >> >> Operations that do not involve data reads or writes would not require > >> >> knowledge of block locations. Applying also the restriction of no > >> >> namespace mutation, an example would be listing directories and > >> >> looking up file information via FileStatus objects (perhaps the only > >> >> examples - its like a safemode but no reads either). > >> >> > >> >> > It is also mentioned, "Use of a BackupNode provides the option of > >> >> > running > >> >> > the NameNode without persistent storage, delegating responsibility > >> >> > for > >> >> > the > >> >> > namespace state persisting to the BackupNode.", what means "running > >> >> > the > >> >> > NameNode without persistent storage" and "delegating responsibility > >> >> > for > >> >> > the > >> >> > namespace state persisting"? > >> >> > >> >> What it means is that the NameNode need not store anything locally, > >> >> but can rely on the edits being stored at the BackupNameNode which > >> >> would continuously be receiving it. When restarted, it can grab a > >> >> current checkpoint from the BNN and boot up anywhere, since there's > no > >> >> local storage requirement. > >> >> > >> >> -- > >> >> Harsh J > >> > > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J > --e89a8f839fc318922804d9c5c1c0 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

I am confused. Hadoopv2 has NN SNN DN JN(journal node), so w= hats
Standby Namenode?

--Send from my Sony mobile.

On Apr 7, 2013 9:03 PM, "Harsh J" <= harsh@cloudera.com> wrote:
BackupNameNode is not present in the maintenance 1.x releases, it is a
feature added to a higher version; you can try it out in 2.x today if
you wish to.

On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
> Hi Harsh,
> Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x?
>
>
> On Sun, Apr 7, 2013 at 4:05 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Yes, it need not keep an edits (transactions) stream locally cause=
>> those are passed synchronously to the BackupNameNode, which persis= ts
>> it on its behalf.
>>
>> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma <linlma@gmail.com> wrote:
>> > Thanks Harsh,
>> >
>> > For your comments, "What it means is that the NameNode n= eed not store
>> > anything locally", you mean Primary Name Node do not nee= d to store
>> > checkpoint/journal locally, and only need to keep memory imag= e
>> > up-to-date
>> > for edits?
>> >
>> > regards,
>> > Lin
>> >
>> >
>> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J <harsh@cloudera.com> wrote:
>> >>
>> >> Hi Lin,
>> >>
>> >> My reply inline.
>> >>
>> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <linlma@gmail.com> wrote:
>> >> > Hi guys,
>> >> >
>> >> > I am reading from this paper to learn about backup n= odes
>> >> > (http://www.storageconference.org/= 2010/Papers/MSST/Shvachko.pdf),
>> >> >
>> >> > It is mentioned, "It contains all file system m= etadata information
>> >> > except
>> >> > for block locations. It can perform all operations o= f the regular
>> >> > NameNode
>> >> > that do not involve modification of the namespace or= knowledge of
>> >> > block
>> >> > locations. ", what kinds of operations do not n= eed knowledge of block
>> >> > locations?
>> >>
>> >> Operations that do not involve data reads or writes would= not require
>> >> knowledge of block locations. Applying also the restricti= on of no
>> >> namespace mutation, an example would be listing directori= es and
>> >> looking up file information via FileStatus objects (perha= ps the only
>> >> examples - its like a safemode but no reads either).
>> >>
>> >> > It is also mentioned, "Use of a BackupNode prov= ides the option of
>> >> > running
>> >> > the NameNode without persistent storage, delegating = responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting to the BackupNode.",= what means "running
>> >> > the
>> >> > NameNode without persistent storage" and "= delegating responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting"?
>> >>
>> >> What it means is that the NameNode need not store anythin= g locally,
>> >> but can rely on the edits being stored at the BackupNameN= ode which
>> >> would continuously be receiving it. When restarted, it ca= n grab a
>> >> current checkpoint from the BNN and boot up anywhere, sin= ce there's no
>> >> local storage requirement.
>> >>
>> >> --
>> >> Harsh J
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>



--
Harsh J
--e89a8f839fc318922804d9c5c1c0--