Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of doducthanhbk@gmail.com
 designates 209.85.216.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=vEvblN2QBNt+pXCqL/mAh64La22tdeJZ6P4azXoAbwrL2HtJpHVYlDgSdJgfZ0ixhE
         beBsWwFMqFfi68lV37pavv0gyzKIcTFIv8cxx/ZyiR6n+6lT19Kq0FZEhmOAf+istqdP
         /MGwAE/JgaEwUhp77wr5ezfGyxkYyUYKVRG8g=
MIME-Version: 1.0
Sender: doducthanhbk@gmail.com
In-Reply-To: <BANLkTi=Pw4yy2y--Y1bLWE-zqBtwzC-W=A@mail.gmail.com>
References: <BANLkTiked1a2zGXLaaErdCWtmSOXbXoCyw@mail.gmail.com>
	<BANLkTi=Pw4yy2y--Y1bLWE-zqBtwzC-W=A@mail.gmail.com>
Date: Fri, 6 May 2011 07:56:22 -0500
Message-ID: <BANLkTi=A3ms3uMas8c+sE7K3nWvTwGwTVQ@mail.gmail.com>
Subject: Re: experience with Backup Node
From: Thanh Do <thanhdo@cs.wisc.edu>
To: hdfs-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=20cf300fb38f4ff03804a29b04ff

--20cf300fb38f4ff03804a29b04ff
Content-Type: text/plain; charset=UTF-8

I see ...

Thanks for useful feedback, Todd!

On Fri, May 6, 2011 at 7:34 AM, Todd Lipcon <todd@cloudera.com> wrote:

> Hi Thanh,
>
> No, I doubt that anybody is running BackupNode in production, since it's
> only part of 0.21, and in my opinion an incomplete implementation. A few of
> the deficiencies I'm aware of:
>
> - Like you said, edits are transferred by synchronous RPC from the NN. As
> far as I know, there are no timeouts enabled on these RPCs, so if the
> backupnode hangs, so will the primary. In the case of a BN crash, the
> primary will hang for many minutes before noticing.
> - The BN doesn't provide hot standby since it doesn't yet receive block
> reports.
>
> The fact that the RPCs are synchronous seems unavoidable if you want to be
> able to do a failover without any lost edits. But without timeouts, it's a
> bit scary.
>
> Some work will be going on in trunk to address high availability over the
> next several months - we'd definitely appreciate your expertise in failure
> injection, etc, being applied to the new code as it goes in!
>
> -Todd
>
> On Thu, May 5, 2011 at 9:28 AM, Thanh Do <thanhdo@cs.wisc.edu> wrote:
>
>> hi all,
>>
>> any body deploy the Backup Node in your system.
>> I am curious about the impact of the Backup Node
>> to the NameNode throughput.
>>
>> To my understanding, NameNode streams edits
>> log operation to the BackupNode (by an RPC call),
>> and only return once that operation has been applied
>> to the in memory state of the Backup Node.
>>
>> Will this RPC call slow down the NameNode a little bit.
>>
>> Thanks
>> Thanh
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

--20cf300fb38f4ff03804a29b04ff
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>I see ...</div><div><br></div>Thanks for useful feedback, Todd!<br><br=
><div class=3D"gmail_quote">On Fri, May 6, 2011 at 7:34 AM, Todd Lipcon <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:todd@cloudera.com">todd@cloudera.com</=
a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Hi Thanh,<div><br></div><div>No, I doubt th=
at anybody is running BackupNode in production, since it&#39;s only part of=
 0.21, and in my opinion an incomplete implementation. A few of the deficie=
ncies I&#39;m aware of:</div>


<div><br></div><div>- Like you said, edits are transferred by synchronous R=
PC from the NN. As far as I know, there are no timeouts enabled on these RP=
Cs, so if the backupnode hangs, so will the primary. In the case of a BN cr=
ash, the primary will hang for many minutes before noticing.</div>


<div>- The BN doesn&#39;t provide hot standby since it doesn&#39;t yet rece=
ive block reports.</div><div><br></div><div>The fact that the RPCs are sync=
hronous seems unavoidable if you want to be able to do a failover without a=
ny lost edits. But without timeouts, it&#39;s a bit scary.</div>


<div><br></div><div>Some work will be going on in trunk to address high ava=
ilability over the next several months - we&#39;d definitely appreciate you=
r expertise in failure injection, etc, being applied to the new code as it =
goes in!</div>


<div><br></div><div>-Todd</div><div><div><div></div><div class=3D"h5"><br><=
div class=3D"gmail_quote">On Thu, May 5, 2011 at 9:28 AM, Thanh Do <span di=
r=3D"ltr">&lt;<a href=3D"mailto:thanhdo@cs.wisc.edu" target=3D"_blank">than=
hdo@cs.wisc.edu</a>&gt;</span> wrote:<br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
hi all,<div><br></div><div>any body deploy the Backup Node in your system.<=
/div><div>I am curious about the impact of=C2=A0the Backup Node</div><div>t=
o the NameNode throughput.</div><div><br></div><div>To my understanding, Na=
meNode streams edits</div>


<div>log operation to the BackupNode (by an RPC call),</div><div>and only r=
eturn once that operation has been applied</div><div>to the in memory state=
 of the Backup Node.</div><div><br></div><div>Will this RPC call slow down =
the NameNode a little bit.</div>


<div><br></div><div>Thanks</div><div>Thanh</div>
</blockquote></div><br><br clear=3D"all"><br></div></div>-- <br>Todd Lipcon=
<br>Software Engineer, Cloudera<br>
</div>
</blockquote></div><br>

--20cf300fb38f4ff03804a29b04ff--