hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: federated cluster
Date Mon, 05 Nov 2012 13:56:13 GMT
Hi Visioner,

If you aren't using HA, you will need two SNNs - one that checkpoints
IP1 and the other for IP2. If you lack more hardware, you may probably
reside them on each other (i.e. SNN of IP2 on IP1, etc.) although this
wouldn't work so well after a while (when metadata at each NN has
grown much), but for starters this is good enough I guess.

The ResourceManager/NodeManager are YARN daemons useful for processing
data. If your need is just storage, you are correct in not needing
them deployed.

You are also correct on the federated NNs part. If a file resides in
the NS of IP1, you need to access IP1 to fetch its details.

On Mon, Nov 5, 2012 at 5:50 PM, Visioner Sadak <visioner.sadak@gmail.com> wrote:
> Hello friends i have configured a 3 node federated hadoop cluster with 2
> namenodes and 3 datanodes
> 2 namenodes -- ip1,ip2
> 3 datanodes   --ip1,ip2,ip3
> wht nodes shud i make as my secondary namenode,is it mandatory to have
> resource manager and node manager,i m using hadoop just for storing purpose
> so i dunt need them
> some data goes into namenode 1 which is ip1, some goes in to namenode 2
> which is ip2
> i hope i cant access the data of nn1 using nn2 ip or any nn ip in my cluster
> right,
> i can access data only using the nn1 ip for nn1 data ....correct me if i am
> wrong...

Harsh J

View raw message