Return-Path: 
 <user-return-14348-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 7A030101F9
	for <apmail-hadoop-hdfs-user-archive@minotaur.apache.org>;
 Sat, 15 Mar 2014 02:53:43 +0000 (UTC)
Received: (qmail 58039 invoked by uid 500); 15 Mar 2014 02:53:35 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 57841 invoked by uid 500); 15 Mar 2014 02:53:34 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 57834 invoked by uid 99); 15 Mar 2014 02:53:34 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 15 Mar 2014 02:53:34 +0000
X-ASF-Spam-Status: No, hits=2.2 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of dlmarion@hotmail.com designates
 65.54.190.87 as permitted sender)
Received: from [65.54.190.87] (HELO bay0-omc2-s12.bay0.hotmail.com)
 (65.54.190.87)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 15 Mar 2014 02:53:27 +0000
Received: from BAY403-EAS51 ([65.54.190.124]) by
 bay0-omc2-s12.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675);
	 Fri, 14 Mar 2014 19:53:05 -0700
X-TMN: [t8v+jGWCg5eROce3QlyNz6AE8lhbX2RT]
X-Originating-Email: [dlmarion@hotmail.com]
Message-ID: <BAY403-EAS51A5F3982099F25430F196D1730@phx.gbl>
Date: Fri, 14 Mar 2014 22:53:03 -0400
Subject: RE: HA NN Failover question
Importance: normal
From: dlmarion <dlmarion@hotmail.com>
To: <user@hadoop.apache.org>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="--_com.android.email_23460460400430"
X-OriginalArrivalTime: 15 Mar 2014 02:53:05.0582 (UTC)
 FILETIME=[B0648CE0:01CF3FF9]
X-Virus-Checked: Checked by ClamAV on apache.org

----_com.android.email_23460460400430
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"

Apache Hadoop 2.3.0


Sent via the Samsung GALAXY S=C2=AE4=2C an AT&T 4G LTE smartphone

-------- Original message --------
From: Azuryy <azuryyyu@gmail.com>
Date:03/14/2014  10:45 PM  (GMT-05:00)
To: user@hadoop.apache.org
Subject: Re: HA NN Failover question

Which Hadoop version you used?


Sent from my iPhone5s

> On 2014=E5=B9=B43=E6=9C=8815=E6=97=A5=2C at 9:29=2C dlmarion <dlmarion@ho=
tmail.com> wrote:
>
> Server 1: NN1 and ZKFC1
> Server 2: NN2 and ZKFC2
> Server 3: Journal1 and ZK1
> Server 4: Journal2 and ZK2
> Server 5: Journal3 and ZK3
> Server 6+: Datanode
>
> All in the same rack. I would expect the ZKFC from the active name node s=
erver to lose its lock and the other ZKFC to tell the standby namenode that=
 it should become active (I=E2=80=99m assuming that=E2=80=99s how it works)=
.
>
> - Dave
>
> From: Juan Carlos [mailto:jucaf1@gmail.com]
> Sent: Friday=2C March 14=2C 2014 9:12 PM
> To: user@hadoop.apache.org
> Subject: Re: HA NN Failover question
>
> Hi Dave=2C
> How many zookeeper servers do you have and where are them?
>
> Juan Carlos Fern=C3=A1ndez Rodr=C3=ADguez
>
> El 15/03/2014=2C a las 01:21=2C dlmarion <dlmarion@hotmail.com> escribi=
=C3=B3:
>
> I was doing some testing with HA NN today. I set up two NN with active fa=
ilover (ZKFC) using sshfence. I tested that its working on both NN by doing=
 =E2=80=98kill -9 <pid>=E2=80=99 on the active NN. When I did this on the a=
ctive node=2C the standby would become the active and everything seemed to =
work. Next=2C I logged onto the active NN and did a =E2=80=98service networ=
k stop=E2=80=99 to simulate a NIC/network failure. The standby did not beco=
me the active in this scenario. In fact=2C it remained in standby mode and =
complained in the log that it could not communicate with (what was) the act=
ive NN. I was unable to find anything relevant via searches in Google in Ji=
ra. Does anyone have experience successfully testing this? I=E2=80=99m hopi=
ng that it is just a configuration problem.
>
> FWIW=2C when the network was restarted on the active NN=2C it failed over=
 almost immediately.
>
> Thanks=2C
>
> Dave

----_com.android.email_23460460400430
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="utf-8"

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html=3B charset=3Dutf-8">
</head>
<body>
<div>Apache Hadoop 2.3.0</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style=3D"font-size:9px=3Bcolor:#575757">Sent via the Samsung GALAXY S=
=C2=AE4=2C an AT&amp=3BT 4G LTE smartphone</div>
</div>
<br>
<br>
-------- Original message --------<br>
From: Azuryy <azuryyyu@gmail.com><br>
Date:03/14/2014 10:45 PM (GMT-05:00) <br>
To: user@hadoop.apache.org <br>
Subject: Re: HA NN Failover question <br>
<br>
<div dir=3D"auto">
<div>Which Hadoop version you used?</div>
<div><br>
<br>
Sent from my iPhone5s</div>
<div><br>
On 2014=E5=B9=B43=E6=9C=8815=E6=97=A5=2C at 9:29=2C dlmarion &lt=3B<a href=
=3D"mailto:dlmarion@hotmail.com">dlmarion@hotmail.com</a>&gt=3B wrote:<br>
<br>
</div>
<blockquote type=3D"cite">
<div><style>
<!--
p.x_MsoNormal=2C li.x_MsoNormal=2C div.x_MsoNormal
	{margin:0in=3B
	margin-bottom:.0001pt=3B
	font-size:11.0pt=3B
	font-family:"Calibri"=2C"sans-serif"}
span.x_MsoHyperlink
	{color:blue=3B
	text-decoration:underline}
span.x_MsoHyperlinkFollowed
	{color:purple=3B
	text-decoration:underline}
p.x_MsoListParagraph=2C li.x_MsoListParagraph=2C div.x_MsoListParagraph
	{margin-top:0in=3B
	margin-right:0in=3B
	margin-bottom:0in=3B
	margin-left:.5in=3B
	margin-bottom:.0001pt=3B
	font-size:11.0pt=3B
	font-family:"Calibri"=2C"sans-serif"}
span.x_EmailStyle17
	{font-family:"Calibri"=2C"sans-serif"=3B
	color:windowtext}
span.x_EmailStyle18
	{font-family:"Calibri"=2C"sans-serif"=3B
	color:#1F497D}
.x_MsoChpDefault
	{font-size:10.0pt}
div.x_WordSection1
	{}
-->
</style>
<div class=3D"x_WordSection1">
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 1: NN1 and ZK=
FC1</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 2: NN2 and ZK=
FC2</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 3: Journal1 a=
nd ZK1</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 4: Journal2 a=
nd ZK2</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 5: Journal3 a=
nd ZK3</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">Server 6&#43=3B: Dat=
anode</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">&nbsp=3B</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">All in the same rack=
. I would expect the ZKFC from the active name node server to lose its lock=
 and the other ZKFC to tell the standby namenode that it should become acti=
ve (I=E2=80=99m assuming that=E2=80=99s how it works).</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">&nbsp=3B</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">- Dave</span></p>
<p class=3D"x_MsoNormal"><span style=3D"color:#1F497D">&nbsp=3B</span></p>
<div>
<div style=3D"border:none=3B border-top:solid #B5C4DF 1.0pt=3B padding:3.0p=
t 0in 0in 0in">
<p class=3D"x_MsoNormal"><b><span style=3D"font-size:10.0pt=3B font-family:=
&quot=3BTahoma&quot=3B=2C&quot=3Bsans-serif&quot=3B">From:</span></b><span =
style=3D"font-size:10.0pt=3B font-family:&quot=3BTahoma&quot=3B=2C&quot=3Bs=
ans-serif&quot=3B"> Juan Carlos [<a href=3D"mailto:jucaf1@gmail.com">mailto=
:jucaf1@gmail.com</a>]
<br>
<b>Sent:</b> Friday=2C March 14=2C 2014 9:12 PM<br>
<b>To:</b> <a href=3D"mailto:user@hadoop.apache.org">user@hadoop.apache.org=
</a><br>
<b>Subject:</b> Re: HA NN Failover question</span></p>
</div>
</div>
<p class=3D"x_MsoNormal">&nbsp=3B</p>
<div>
<p class=3D"x_MsoNormal">Hi Dave=2C</p>
</div>
<div>
<p class=3D"x_MsoNormal">How many zookeeper servers do you have and where a=
re them?&nbsp=3B</p>
</div>
<div>
<p class=3D"x_MsoNormal"><br>
Juan Carlos Fern=C3=A1ndez Rodr=C3=ADguez</p>
</div>
<div>
<p class=3D"x_MsoNormal" style=3D"margin-bottom:12.0pt"><br>
El 15/03/2014=2C a las 01:21=2C dlmarion &lt=3B<a href=3D"mailto:dlmarion@h=
otmail.com">dlmarion@hotmail.com</a>&gt=3B escribi=C3=B3:</p>
</div>
<blockquote style=3D"margin-top:5.0pt=3B margin-bottom:5.0pt">
<div>
<p class=3D"x_MsoNormal">I was doing some testing with HA NN today. I set u=
p two NN with active failover (ZKFC) using sshfence. I tested that its work=
ing on both NN by doing =E2=80=98kill -9 &lt=3Bpid&gt=3B=E2=80=99 on the ac=
tive NN. When I did this on the active node=2C the standby would
 become the active and everything seemed to work. Next=2C I logged onto the=
 active NN and did a =E2=80=98service network stop=E2=80=99 to simulate a N=
IC/network failure. The standby did not become the active in this scenario.=
 In fact=2C it remained in standby mode and complained
 in the log that it could not communicate with (what was) the active NN. I =
was unable to find anything relevant via searches in Google in Jira. Does a=
nyone have experience successfully testing this? I=E2=80=99m hoping that it=
 is just a configuration problem.</p>
<p class=3D"x_MsoNormal">&nbsp=3B</p>
<p class=3D"x_MsoNormal">FWIW=2C when the network was restarted on the acti=
ve NN=2C it failed over almost immediately.</p>
<p class=3D"x_MsoNormal">&nbsp=3B</p>
<p class=3D"x_MsoNormal">Thanks=2C</p>
<p class=3D"x_MsoNormal">&nbsp=3B</p>
<p class=3D"x_MsoNormal">Dave</p>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</body>
</html>

----_com.android.email_23460460400430--