hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thanh Do <than...@cs.wisc.edu>
Subject Re: experience with Backup Node
Date Fri, 06 May 2011 12:56:22 GMT
I see ...

Thanks for useful feedback, Todd!

On Fri, May 6, 2011 at 7:34 AM, Todd Lipcon <todd@cloudera.com> wrote:

> Hi Thanh,
>
> No, I doubt that anybody is running BackupNode in production, since it's
> only part of 0.21, and in my opinion an incomplete implementation. A few of
> the deficiencies I'm aware of:
>
> - Like you said, edits are transferred by synchronous RPC from the NN. As
> far as I know, there are no timeouts enabled on these RPCs, so if the
> backupnode hangs, so will the primary. In the case of a BN crash, the
> primary will hang for many minutes before noticing.
> - The BN doesn't provide hot standby since it doesn't yet receive block
> reports.
>
> The fact that the RPCs are synchronous seems unavoidable if you want to be
> able to do a failover without any lost edits. But without timeouts, it's a
> bit scary.
>
> Some work will be going on in trunk to address high availability over the
> next several months - we'd definitely appreciate your expertise in failure
> injection, etc, being applied to the new code as it goes in!
>
> -Todd
>
> On Thu, May 5, 2011 at 9:28 AM, Thanh Do <thanhdo@cs.wisc.edu> wrote:
>
>> hi all,
>>
>> any body deploy the Backup Node in your system.
>> I am curious about the impact of the Backup Node
>> to the NameNode throughput.
>>
>> To my understanding, NameNode streams edits
>> log operation to the BackupNode (by an RPC call),
>> and only return once that operation has been applied
>> to the in memory state of the Backup Node.
>>
>> Will this RPC call slow down the NameNode a little bit.
>>
>> Thanks
>> Thanh
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message