Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DCA68F7A9 for ; Sun, 7 Apr 2013 14:02:50 +0000 (UTC) Received: (qmail 6204 invoked by uid 500); 7 Apr 2013 14:02:46 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 5853 invoked by uid 500); 7 Apr 2013 14:02:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 5846 invoked by uid 99); 7 Apr 2013 14:02:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 14:02:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of azuryyyu@gmail.com designates 209.85.223.170 as permitted sender) Received: from [209.85.223.170] (HELO mail-ie0-f170.google.com) (209.85.223.170) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 14:02:41 +0000 Received: by mail-ie0-f170.google.com with SMTP id c11so5930925ieb.1 for ; Sun, 07 Apr 2013 07:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=Qi2MLJCPdtbgkpcVlOvJa5pwsqHc5TFLRTlI/kBBM9M=; b=IlaxtXYCEpStFR43F2TVO4D1c4YIIxrVc5DP0wdP6X+N0aSnT6b6vmF78lCe/dCIJb u+WPzhHRrlFsCJVIi58V4W8aEACrY+C8nzJK3eQo+Cd8CxTdUW0BJTh7NuGTNW7F7Tc5 ogLQplUIkcl3qF4x+yRLPdkt84nekXnWOsquaWn5oY3EL0GP6PTwNkIaYzmZ7v4bp+yY yvgvErEO4P+TZE0nV/ZP5WIw4PiKLCRARVt7xsyLfmcdRR3Xk516k6JUualIMCGtdfBC uhg3cwC5rGYbo7bNY6fWOBAQIzh9KYemwyz/QWC/pcYgwaV9Nhk4euLq79UOB4Ke2hYq IHhQ== MIME-Version: 1.0 X-Received: by 10.50.216.231 with SMTP id ot7mr4342762igc.84.1365343341417; Sun, 07 Apr 2013 07:02:21 -0700 (PDT) Received: by 10.64.8.7 with HTTP; Sun, 7 Apr 2013 07:02:21 -0700 (PDT) Received: by 10.64.8.7 with HTTP; Sun, 7 Apr 2013 07:02:21 -0700 (PDT) In-Reply-To: References: Date: Sun, 7 Apr 2013 22:02:21 +0800 Message-ID: Subject: Re: backup node question From: Azuryy Yu To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=14dae93406cddc9a6604d9c5c48f X-Virus-Checked: Checked by ClamAV on apache.org --14dae93406cddc9a6604d9c5c48f Content-Type: text/plain; charset=ISO-8859-1 SNN=secondary name node in my last mail. --Send from my Sony mobile. On Apr 7, 2013 10:01 PM, "Azuryy Yu" wrote: > I am confused. Hadoopv2 has NN SNN DN JN(journal node), so whats > Standby Namenode? > > --Send from my Sony mobile. > On Apr 7, 2013 9:03 PM, "Harsh J" wrote: > >> BackupNameNode is not present in the maintenance 1.x releases, it is a >> feature added to a higher version; you can try it out in 2.x today if >> you wish to. >> >> On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu wrote: >> > Hi Harsh, >> > Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x? >> > >> > >> > On Sun, Apr 7, 2013 at 4:05 PM, Harsh J wrote: >> >> >> >> Yes, it need not keep an edits (transactions) stream locally cause >> >> those are passed synchronously to the BackupNameNode, which persists >> >> it on its behalf. >> >> >> >> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma wrote: >> >> > Thanks Harsh, >> >> > >> >> > For your comments, "What it means is that the NameNode need not store >> >> > anything locally", you mean Primary Name Node do not need to store >> >> > checkpoint/journal locally, and only need to keep memory image >> >> > up-to-date >> >> > for edits? >> >> > >> >> > regards, >> >> > Lin >> >> > >> >> > >> >> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J wrote: >> >> >> >> >> >> Hi Lin, >> >> >> >> >> >> My reply inline. >> >> >> >> >> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma wrote: >> >> >> > Hi guys, >> >> >> > >> >> >> > I am reading from this paper to learn about backup nodes >> >> >> > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf), >> >> >> > >> >> >> > It is mentioned, "It contains all file system metadata information >> >> >> > except >> >> >> > for block locations. It can perform all operations of the regular >> >> >> > NameNode >> >> >> > that do not involve modification of the namespace or knowledge of >> >> >> > block >> >> >> > locations. ", what kinds of operations do not need knowledge of >> block >> >> >> > locations? >> >> >> >> >> >> Operations that do not involve data reads or writes would not >> require >> >> >> knowledge of block locations. Applying also the restriction of no >> >> >> namespace mutation, an example would be listing directories and >> >> >> looking up file information via FileStatus objects (perhaps the only >> >> >> examples - its like a safemode but no reads either). >> >> >> >> >> >> > It is also mentioned, "Use of a BackupNode provides the option of >> >> >> > running >> >> >> > the NameNode without persistent storage, delegating responsibility >> >> >> > for >> >> >> > the >> >> >> > namespace state persisting to the BackupNode.", what means >> "running >> >> >> > the >> >> >> > NameNode without persistent storage" and "delegating >> responsibility >> >> >> > for >> >> >> > the >> >> >> > namespace state persisting"? >> >> >> >> >> >> What it means is that the NameNode need not store anything locally, >> >> >> but can rely on the edits being stored at the BackupNameNode which >> >> >> would continuously be receiving it. When restarted, it can grab a >> >> >> current checkpoint from the BNN and boot up anywhere, since there's >> no >> >> >> local storage requirement. >> >> >> >> >> >> -- >> >> >> Harsh J >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Harsh J >> > >> > >> >> >> >> -- >> Harsh J >> > --14dae93406cddc9a6604d9c5c48f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

SNN=3Dsecondary name node=A0 in my last mail.

--Send from my Sony mobile.

On Apr 7, 2013 10:01 PM, "Azuryy Yu" &= lt;azuryyyu@gmail.com> wrote:<= br type=3D"attribution">

I am confused. Hadoopv2 has NN SNN DN JN(journal node), so w= hats
Standby Namenode?

--Send from my Sony mobile.

On Apr 7, 2013 9:03 PM, "Harsh J" <= harsh@cloudera.com<= /a>> wrote:
BackupNameNode is not present in the maintenance 1.x releases, it is a
feature added to a higher version; you can try it out in 2.x today if
you wish to.

On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
> Hi Harsh,
> Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x?
>
>
> On Sun, Apr 7, 2013 at 4:05 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Yes, it need not keep an edits (transactions) stream locally cause=
>> those are passed synchronously to the BackupNameNode, which persis= ts
>> it on its behalf.
>>
>> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma <linlma@gmail.com> wrote:
>> > Thanks Harsh,
>> >
>> > For your comments, "What it means is that the NameNode n= eed not store
>> > anything locally", you mean Primary Name Node do not nee= d to store
>> > checkpoint/journal locally, and only need to keep memory imag= e
>> > up-to-date
>> > for edits?
>> >
>> > regards,
>> > Lin
>> >
>> >
>> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J <harsh@cloudera.com> wrote: >> >>
>> >> Hi Lin,
>> >>
>> >> My reply inline.
>> >>
>> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <linlma@gmail.com> wrote: >> >> > Hi guys,
>> >> >
>> >> > I am reading from this paper to learn about backup n= odes
>> >> > (http://www.storageconference.org/= 2010/Papers/MSST/Shvachko.pdf),
>> >> >
>> >> > It is mentioned, "It contains all file system m= etadata information
>> >> > except
>> >> > for block locations. It can perform all operations o= f the regular
>> >> > NameNode
>> >> > that do not involve modification of the namespace or= knowledge of
>> >> > block
>> >> > locations. ", what kinds of operations do not n= eed knowledge of block
>> >> > locations?
>> >>
>> >> Operations that do not involve data reads or writes would= not require
>> >> knowledge of block locations. Applying also the restricti= on of no
>> >> namespace mutation, an example would be listing directori= es and
>> >> looking up file information via FileStatus objects (perha= ps the only
>> >> examples - its like a safemode but no reads either).
>> >>
>> >> > It is also mentioned, "Use of a BackupNode prov= ides the option of
>> >> > running
>> >> > the NameNode without persistent storage, delegating = responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting to the BackupNode.",= what means "running
>> >> > the
>> >> > NameNode without persistent storage" and "= delegating responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting"?
>> >>
>> >> What it means is that the NameNode need not store anythin= g locally,
>> >> but can rely on the edits being stored at the BackupNameN= ode which
>> >> would continuously be receiving it. When restarted, it ca= n grab a
>> >> current checkpoint from the BNN and boot up anywhere, sin= ce there's no
>> >> local storage requirement.
>> >>
>> >> --
>> >> Harsh J
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>



--
Harsh J
--14dae93406cddc9a6604d9c5c48f--