Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DD52A10972 for ; Thu, 12 Feb 2015 12:02:37 +0000 (UTC) Received: (qmail 116 invoked by uid 500); 12 Feb 2015 12:02:21 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 99918 invoked by uid 500); 12 Feb 2015 12:02:21 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 99903 invoked by uid 99); 12 Feb 2015 12:02:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Feb 2015 12:02:21 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shekhar.kotekar@gmail.com designates 209.85.217.181 as permitted sender) Received: from [209.85.217.181] (HELO mail-lb0-f181.google.com) (209.85.217.181) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Feb 2015 12:02:16 +0000 Received: by mail-lb0-f181.google.com with SMTP id b6so9002276lbj.12 for ; Thu, 12 Feb 2015 04:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=uXqqjwjcbU1XUGngdcpifDqNNceQr+skHiRZLG7rUdo=; b=g9MdaLTj1fWtnSC637/hC9jx3RTgwRZm6C1AUPGD1cYpbggZ5vKy0ssXlF7muVrVo9 P34Rck9kEZCcN5Do2eGQN31OO3zAgB50NsUlW9m6Vr7mG+t+/Zwmqi41ra8s/0InnsEh soUDFRxwSANBmsj5DPw96A27H6i0pb9KH3mEpbkbwak72egDNZRHmNxYNy3j0bhqDbHm XugPDAVQKQj8sgGQ+B5tyAelnzvht1qD7uuakMEQE8xL9bgIHhE1upHMCbMDYAccFJiu BQKsjb7znoamEVGfoeVtongwCSKI69n1Ckrhtv9xAqAPcEuDI/b4IPhyhmEhX2Tdkpqu hsVg== X-Received: by 10.152.179.135 with SMTP id dg7mr3051351lac.58.1423742515477; Thu, 12 Feb 2015 04:01:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.3.7 with HTTP; Thu, 12 Feb 2015 04:01:35 -0800 (PST) In-Reply-To: <8AD4EE147886274A8B495D6AF407DF698122B8E4@szxeml510-mbx.china.huawei.com> References: <8AD4EE147886274A8B495D6AF407DF698122B8E4@szxeml510-mbx.china.huawei.com> From: Chandrashekhar Kotekar Date: Thu, 12 Feb 2015 17:31:35 +0530 Message-ID: Subject: Re: journal node shared edits directory should be present on HDFS or NAS or anything else? To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11349ae6e30270050ee2e2c6 X-Virus-Checked: Checked by ClamAV on apache.org --001a11349ae6e30270050ee2e2c6 Content-Type: text/plain; charset=UTF-8 Hi Brahma Reddy, Thanks for the quick answer. It explains a lot but I have one more question. Maybe it is a stupid question but, "required shared storage" means active name node will write to its local disk? Do I need to configure or use any shared storage like NAS or SAN array or S3 storage for this purpose? Regards, Chandrash3khar Kotekar Mobile - +91 8600011455 On Thu, Feb 12, 2015 at 5:08 PM, Brahma Reddy Battula < brahmareddy.battula@huawei.com> wrote: > Hello Chandrashekhar, > > Active namenode will write to require shared storage and will not write > to HDFS.. Please check following docs for reference > > > > *When Sharedstorage is Journalnode:* > > > dfs.namenode.shared.edits.dir > qjournal://node1.example.com:8485;node2.example.com:8485;node3.example.com:8485/mycluster > > > > > http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html > > > > *When Sharedstorage is NFS:* > > > dfs.namenode.shared.edits.dir > file:///mnt/filer1/dfs/ha-name-dir-shared > > > > > > http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html > > > > > > Thanks & Regards > > Brahma Reddy Battula > > > ------------------------------ > *From:* Chandrashekhar Kotekar [shekhar.kotekar@gmail.com] > *Sent:* Thursday, February 12, 2015 5:01 PM > *To:* user@hadoop.apache.org > *Subject:* journal node shared edits directory should be present on HDFS > or NAS or anything else? > > Hi, > > I am trying to configure name node HA and I want to further configure > automatic fail over. I am confused about '*dfs.namenode.shared.edits.dir*' > configuration. > > Documentation says that "active namde node writes to shared storage." I > would like to know if this means that name nodes write it on HDFS or do > they require shared storage like NAS or SAN or something else. > > > Regards, > Chandrash3khar Kotekar > Mobile - +91 8600011455 > --001a11349ae6e30270050ee2e2c6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Brahma Reddy,

Thanks for the quick a= nswer. It explains a lot but I have one more question. Maybe it is a stupid= question but, "re= quired shared storage" means active name node will write to its= local disk? Do I need to configure or use any shared storage like NAS or S= AN array or S3 storage for this purpose?

=
Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455

On Thu, Feb 12, 2015 at 5:08 PM, Brahma Redd= y Battula <brahmareddy.battula@huawei.com> wrot= e:
Hello Chandrashekhar,

Active namenode will write to require shared storag= e and will not write to HDFS.. Please check following doc= s for reference



When Sharedstorage is Journalno= de:

<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://node1.e=
xample.com:8485;node2.example=
.com:8485;node3.example.com:8485/mycluster</value>
</property>

http://hadoop.apac= he.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWith= QJM.html



When Sharedstorage is NFS:<= /font>

<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>file:///mnt/filer1/dfs/ha-name-dir-shared</value>
</property>


http://hadoop.apac= he.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWith= NFS.html





Thanks & Regards

=C2=A0Brahma Reddy Battula

=C2=A0

Fro= m: Chandrashekhar Kotekar [shekhar.kotekar@gmail.com]
Sent: Thursday, February 12, 2015 5:01 PM
To: user= @hadoop.apache.org
Subject: journal node shared edits directory should be present on HD= FS or NAS or anything else?

Hi,

I am =C2=A0trying to configure name node HA and I want to further conf= igure automatic fail over. I am confused about 'dfs.namenode.shared.edits.dir' configuration.

Documentation says that "active namde node writes to shared stora= ge." I would like to know if this means that name nodes write it on HD= FS or do they require shared storage like NAS or SAN or something else.


Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455

--001a11349ae6e30270050ee2e2c6--