Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 032D1FFB4 for ; Sun, 7 Apr 2013 07:51:58 +0000 (UTC) Received: (qmail 4250 invoked by uid 500); 7 Apr 2013 07:51:53 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 3962 invoked by uid 500); 7 Apr 2013 07:51:53 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 3948 invoked by uid 99); 7 Apr 2013 07:51:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 07:51:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of linlma@gmail.com designates 209.85.192.175 as permitted sender) Received: from [209.85.192.175] (HELO mail-pd0-f175.google.com) (209.85.192.175) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Apr 2013 07:51:48 +0000 Received: by mail-pd0-f175.google.com with SMTP id g10so2467726pdj.20 for ; Sun, 07 Apr 2013 00:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=QyCp73o3FdsOlhnRH6+o4FZIPliQStcXGn8UV2TIUaQ=; b=QWlsTYab6LBAHlbqqjx11NbkoQZqw0yPX7b3w+u1JkXiteeBUyiBCkYBolBEB55lpL QJLIr8euYQI7KBsn8Ufxsi6o8BM+IdIp/bsMG3FkmBDp6Et4liskqQb9/+ERLfc/HLo3 zu7S3d3HElSIaAIvrQE9VFJwJW3Ey4Mg48nE5fRsyv5Rin0MjIUFkTZ2/9l7OZ1m13IB szUbPhfckbu2Wze2oyCd5VeizmoMozf97fmRiroTQvBfkdpEzzTW3G4bsR1po9F+9HFY eAW+T5GVGKUqiSw5x/lKyC6SUD+R1qHx+x+jlSiNbmlQqZiHROH7EAmV6INKcRvUPyDb GwSg== MIME-Version: 1.0 X-Received: by 10.66.139.198 with SMTP id ra6mr25911042pab.148.1365321088229; Sun, 07 Apr 2013 00:51:28 -0700 (PDT) Received: by 10.66.219.201 with HTTP; Sun, 7 Apr 2013 00:51:28 -0700 (PDT) In-Reply-To: References: Date: Sun, 7 Apr 2013 15:51:28 +0800 Message-ID: Subject: Re: backup node question From: Lin Ma To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b5dbb7077e48e04d9c09670 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5dbb7077e48e04d9c09670 Content-Type: text/plain; charset=ISO-8859-1 Thanks Harsh, For your comments, "What it means is that the NameNode need not store anything locally", you mean Primary Name Node do not need to store checkpoint/journal locally, and only need to keep memory image up-to-date for edits? regards, Lin On Sun, Apr 7, 2013 at 3:31 PM, Harsh J wrote: > Hi Lin, > > My reply inline. > > On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma wrote: > > Hi guys, > > > > I am reading from this paper to learn about backup nodes > > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf), > > > > It is mentioned, "It contains all file system metadata information except > > for block locations. It can perform all operations of the regular > NameNode > > that do not involve modification of the namespace or knowledge of block > > locations. ", what kinds of operations do not need knowledge of block > > locations? > > Operations that do not involve data reads or writes would not require > knowledge of block locations. Applying also the restriction of no > namespace mutation, an example would be listing directories and > looking up file information via FileStatus objects (perhaps the only > examples - its like a safemode but no reads either). > > > It is also mentioned, "Use of a BackupNode provides the option of running > > the NameNode without persistent storage, delegating responsibility for > the > > namespace state persisting to the BackupNode.", what means "running the > > NameNode without persistent storage" and "delegating responsibility for > the > > namespace state persisting"? > > What it means is that the NameNode need not store anything locally, > but can rely on the edits being stored at the BackupNameNode which > would continuously be receiving it. When restarted, it can grab a > current checkpoint from the BNN and boot up anywhere, since there's no > local storage requirement. > > -- > Harsh J > --047d7b5dbb7077e48e04d9c09670 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks Harsh,

For your comments, "What it means is that the Nam= eNode need not store anything locally", you mean Primary Name Node do = not need to store checkpoint/journal locally, and only need to keep memory = image up-to-date for edits?

regards,
Lin

On Sun, Apr 7, 2013 a= t 3:31 PM, Harsh J <harsh@cloudera.com> wrote:
Hi Lin,

My reply inline.

On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <linlma@gmail.com> wrote:
> Hi guys,
>
> I am reading from this paper to learn about backup nodes
> (http://www.storageconference.org/2010/Papers/MSST/S= hvachko.pdf),
>
> It is mentioned, "It contains all file system metadata informatio= n except
> for block locations. It can perform all operations of the regular Name= Node
> that do not involve modification of the namespace or knowledge of bloc= k
> locations. ", what kinds of operations do not need knowledge of b= lock
> locations?

Operations that do not involve data reads or writes would not require=
knowledge of block locations. Applying also the restriction of no
namespace mutation, an example would be listing directories and
looking up file information via FileStatus objects (perhaps the only
examples - its like a safemode but no reads either).

> It is also mentioned, "Use of a BackupNode provides the option of= running
> the NameNode without persistent storage, delegating responsibility for= the
> namespace state persisting to the BackupNode.", what means "= running the
> NameNode without persistent storage" and "delegating respons= ibility for the
> namespace state persisting"?

What it means is that the NameNode need not store anything locally, but can rely on the edits being stored at the BackupNameNode which
would continuously be receiving it. When restarted, it can grab a
current checkpoint from the BNN and boot up anywhere, since there's no<= br> local storage requirement.

--
Harsh J

--047d7b5dbb7077e48e04d9c09670--