hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GOEKE, MATTHEW (AG/1000)" <matthew.go...@monsanto.com>
Subject RE: hadoop high availability using terracotta
Date Mon, 26 Sep 2011 02:42:55 GMT
I don't have much terracotta experience but which solution of theirs are you referring to?
After looking at their product page I am having a hard time figuring out what architecture
you are planning to build that would have a faster turnaround on failure.


-----Original Message-----
From: Gokul Prasad [mailto:nhgokulprasad@gmail.com] 
Sent: Saturday, September 24, 2011 11:33 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop high availability using terracotta

afaik..drbd approach will take longer to bring standby namenode to
live..am trying to minimise the downtime..like standby namenode will
be relayed with the necessary info thro terracotta and if primary
namenode goes down standby will take its role.


On Sat, Sep 24, 2011 at 1:00 AM, GOEKE, MATTHEW (AG/1000)
<matthew.goeke@monsanto.com> wrote:
> Is there any reason not to just use something as simple as drbd (which is baked into
vanilla linux)? Unless you are already paying terracotta for their software there are plenty
of HA setups that are supported through the use of drbd and a virtual ip.
> Matt
> -----Original Message-----
> From: Gokul Prasad [mailto:nhgokulprasad@gmail.com]
> Sent: Friday, September 23, 2011 2:06 PM
> To: hdfs-user@hadoop.apache.org
> Subject: Re: hadoop high availability using terracotta
> Cos,
> Am planning to implement this idea in bigger cluster but first i want
> to get confidence in small cluster. I do read about terracotta and
> inspired by this idea.
> -gokul
> On Fri, Sep 23, 2011 at 11:57 PM, Konstantin Boudnik <cos@apache.org> wrote:
>> Gokul,
>> Terracotta is a fascinating framework which might be a bit of an overshot for a
>> small setup as well as adding another level of complexity in the form
>> Terracotta framework.
>> Quite interesting approach for NN's HA is Jini (aka ASF River) along with
>> JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
>> you for free such benefits as auto-discovering making hot-swap of NN a non-issue.
>> I've been musing over this idea for some time and am trying to find some time
>> to actually experiment with it.
>> Cos
>> On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>>>    Hi,
>>>    Am trying to bring high availability in namenode(in hadoop cluster) for
>>>    small setup.
>>>    So the idea is to keep one extra namenode and use terracotta to replicate
>>>    the log(in memory details-jvm) from primary datacode to standby namenode,
>>>    if primary goes down secondary can take the primary role.
>>>    Now am bit confused how to start second namenode while the primary name
>>>    node up and running(with datanode connected) with keep relying the info
>>>    from primary thro terracotta. Is there a way to achieve this without
>>>    restarting namenode or datanode, any patch?
>>>    Please help and suggest me.
>>>    Regards,
>>>    Gokul
> This e-mail message may contain privileged and/or confidential information, and is intended
to be received only by persons entitled
> to receive such information. If you have received this e-mail in error, please notify
the sender immediately. Please delete it and
> all attachments from any servers, hard drives or any other media. Other use of this e-mail
by you is strictly prohibited.
> All e-mails and attachments sent and received are subject to monitoring, reading and
archival by Monsanto, including its
> subsidiaries. The recipient of this e-mail is solely responsible for checking for the
presence of "Viruses" or other "Malware".
> Monsanto, along with its subsidiaries, accepts no liability for any damage caused by
any such code transmitted by or accompanying
> this e-mail or any attachment.
> The information contained in this email may be subject to the export control laws and
regulations of the United States, potentially
> including but not limited to the Export Administration Regulations (EAR) and sanctions
regulations issued by the U.S. Department of
> Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information
you are obligated to comply with all
> applicable U.S. export laws and regulations.

View raw message