Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of nitinpawar432@gmail.com
 designates 209.85.217.173 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHg+sbPp1-1P3eOvi7grp0CzSHHTUpkjfJo0UyXuaMfhsXpcAA@mail.gmail.com>
References: 
 <CAHg+sbPp1-1P3eOvi7grp0CzSHHTUpkjfJo0UyXuaMfhsXpcAA@mail.gmail.com>
Date: Mon, 6 May 2013 14:30:49 +0530
Message-ID: 
 <CAORpBshXnj-NYCS8BR=teTL8o9f_DNwAsLA2b9ocsqROE34ZMA@mail.gmail.com>
Subject: Re: Namenode going to safe mode on YARN
From: Nitin Pawar <nitinpawar432@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=089e0158c738e07dc804dc08efa4

--089e0158c738e07dc804dc08efa4
Content-Type: text/plain; charset=ISO-8859-1

What is your replication factor on hdfs?
Did any of your datanode go down recently and is not back in rotation?
Did you delete any hdfs blocks directly from datanodes?
On May 6, 2013 2:28 PM, "Krishna Kishore Bonagiri" <write2kishore@gmail.com>
wrote:

> Hi,
>
>   I have been running application on my YARN cluster since around 20 days,
> about 5000 applications a day. I am getting the following error today.
> Please let me know how can I avoid this, is this happening because of a bug?
>
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException):
> Cannot create file/1066/AppMaster.jar. Name node is in safe mode.
> The reported blocks 4775 needs additional 880 blocks to reach the
> threshold 0.9990 of total blocks 5660. Safe mode will be turned off
> automatically.
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1786)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:1737)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1719)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:429)
>         at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:271)
>         at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:40732)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1737)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1735)
>
>
> Thanks,
> Kishore
>

--089e0158c738e07dc804dc08efa4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr">What is your replication factor on hdfs?<br>
Did any of your datanode go down recently and is not back in rotation?<br>
Did you delete any hdfs blocks directly from datanodes?</p>
<div class=3D"gmail_quote">On May 6, 2013 2:28 PM, &quot;Krishna Kishore Bo=
nagiri&quot; &lt;<a href=3D"mailto:write2kishore@gmail.com">write2kishore@g=
mail.com</a>&gt; wrote:<br type=3D"attribution"><blockquote class=3D"gmail_=
quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1=
ex">
<div dir=3D"ltr">Hi,=A0<div><br><div>=A0 I have been running application on=
 my YARN cluster since around 20 days, about 5000 applications a day. I am =
getting the following error today. Please let me know how can I avoid this,=
 is this happening because of a bug?</div>

<div><br></div><div><div>org.apache.hadoop.ipc.RemoteException(org.apache.h=
adoop.hdfs.server.namenode.SafeModeException): Cannot create file/1066/AppM=
aster.jar. Name node is in safe mode.</div><div>The reported blocks 4775 ne=
eds additional 880 blocks to reach the threshold 0.9990 of total blocks 566=
0. Safe mode will be turned off automatically.</div>

<div>=A0 =A0 =A0 =A0 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem=
.startFileInternal(FSNamesystem.java:1786)</div><div>=A0 =A0 =A0 =A0 at org=
.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.=
java:1737)</div>

<div>=A0 =A0 =A0 =A0 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem=
.startFile(FSNamesystem.java:1719)</div><div>=A0 =A0 =A0 =A0 at org.apache.=
hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java=
:429)</div>
<div>
=A0 =A0 =A0 =A0 at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocol=
ServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.=
java:271)</div><div>=A0 =A0 =A0 =A0 at org.apache.hadoop.hdfs.protocol.prot=
o.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(=
ClientNamenodeProtocolProtos.java:40732)</div>

<div>=A0 =A0 =A0 =A0 at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$Prot=
oBufRpcInvoker.call(ProtobufRpcEngine.java:454)</div><div>=A0 =A0 =A0 =A0 a=
t org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)</div><div>=A0 =A0 =
=A0 =A0 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741)</di=
v>

<div>=A0 =A0 =A0 =A0 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.j=
ava:1737)</div><div>=A0 =A0 =A0 =A0 at java.security.AccessController.doPri=
vileged(Native Method)</div><div>=A0 =A0 =A0 =A0 at javax.security.auth.Sub=
ject.doAs(Subject.java:415)</div>

<div>=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGroupInformation.doA=
s(UserGroupInformation.java:1478)</div><div>=A0 =A0 =A0 =A0 at org.apache.h=
adoop.ipc.Server$Handler.run(Server.java:1735)</div><div><br></div><div><br=
></div><div>

Thanks,</div><div>Kishore</div></div></div></div>
</blockquote></div>

--089e0158c738e07dc804dc08efa4--