hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: journal node shared edits directory should be present on HDFS or NAS or anything else?
Date Thu, 12 Feb 2015 12:20:57 GMT
Hello Chandrashekhar,

Yes, you need to configure the shared storage(Active namenode writes to shared storage and
Standby NN will read).

 Please check following mail for configuration..Shared storage can be Journalnode(which is
one process and come along with hadoop package, check following reference for more details)/NFS/BOOKKeeper





Thanks & Regards

 Brahma Reddy Battula




________________________________
From: Chandrashekhar Kotekar [shekhar.kotekar@gmail.com]
Sent: Thursday, February 12, 2015 5:31 PM
To: user@hadoop.apache.org
Subject: Re: journal node shared edits directory should be present on HDFS or NAS or anything
else?

Hi Brahma Reddy,

Thanks for the quick answer. It explains a lot but I have one more question. Maybe it is a
stupid question but, "required shared storage" means active name node will write to its local
disk? Do I need to configure or use any shared storage like NAS or SAN array or S3 storage
for this purpose?


Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455

On Thu, Feb 12, 2015 at 5:08 PM, Brahma Reddy Battula <brahmareddy.battula@huawei.com<mailto:brahmareddy.battula@huawei.com>>
wrote:
Hello Chandrashekhar,

Active namenode will write to require shared storage and will not write to HDFS.. Please check
following docs for reference



When Sharedstorage is Journalnode:


<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://node1.example.com:8485<http://node1.example.com:8485>;node2.example.com:8485<http://node2.example.com:8485>;node3.example.com:8485/mycluster<http://node3.example.com:8485/mycluster></value>
</property>

http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html



When Sharedstorage is NFS:


<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>file:///mnt/filer1/dfs/ha-name-dir-shared</value>
</property>


http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html






Thanks & Regards

 Brahma Reddy Battula



________________________________
From: Chandrashekhar Kotekar [shekhar.kotekar@gmail.com<mailto:shekhar.kotekar@gmail.com>]
Sent: Thursday, February 12, 2015 5:01 PM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: journal node shared edits directory should be present on HDFS or NAS or anything
else?

Hi,

I am  trying to configure name node HA and I want to further configure automatic fail over.
I am confused about 'dfs.namenode.shared.edits.dir' configuration.

Documentation says that "active namde node writes to shared storage." I would like to know
if this means that name nodes write it on HDFS or do they require shared storage like NAS
or SAN or something else.


Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455<tel:%2B91%208600011455>


Mime
View raw message