hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Segel <michael_se...@hotmail.com>
Subject Re: One NameNode keeps Rolling Edit Log on HDFS Federation
Date Sun, 24 Feb 2013 17:53:44 GMT
I think part of the confusion stems from the fact that federation of name nodes only splits
the very large cluster in to smaller portions of the same cluster. If you lose a federated
name node, you only lose a portion of the cluster not the whole thing. So now instead of one
SPOF, you have two SPOFs. 

The advantage of Federation is that you reduce the amount of memory required for the NN.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 24, 2013, at 10:48 AM, Harsh J <harsh@cloudera.com> wrote:

> Hi,
> 
> Federated namenodes are independent of one another (except that they
> both get reports from all the/common DNs in the cluster). It is
> natural to see one roll its edit logs based on its own rate of
> metadata growth, as compared to the other. Their edits, image, etc.
> everything is independent - they also do not know about/talk to each
> other.
> 
> Given the above, I'd say (1) is true and (2) is invalid, and (3) can
> be found in PDFs attached to
> https://issues.apache.org/jira/browse/HDFS-1052.
> 
> On Sun, Feb 24, 2013 at 8:39 PM, YouPeng Yang <yypvsxf19870706@gmail.com> wrote:
>> Hi All
>> 
>> I'm testing the HDFS Federation.I have 2 namenodes in  my cluster.
>> I find that it is  Rolling Edit Log continuously on one   namenode,however
>> the other one  changes nothing.
>> 
>> My question :
>> 1. Is it the right situation.
>> 2. I have thought that the two namenodes shoud keep concurrency.
>> why they get differences.
>> 3. is there any advanced docs about HDFS Federation.
>> 
>> thanks.
>> 
>> Regards.
> 
> 
> 
> --
> Harsh J
> 

Mime
View raw message