Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of samliuhadoop@gmail.com
 designates 209.85.216.176 as permitted sender)
MIME-Version: 1.0
Date: Sun, 19 Oct 2014 19:56:04 -0700
Message-ID: 
 <CAHH8OOfJvwiP_xOBg+D6mzKW=3S7Oqb7hg1AaU9dCyDGcxCPsQ@mail.gmail.com>
Subject: Can add a regular check in DataNode on free disk space?
From: sam liu <samliuhadoop@gmail.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7b66fbab0b70200505d1db5d

--047d7b66fbab0b70200505d1db5d
Content-Type: text/plain; charset=UTF-8

Hi Experts and Developers,

At present, if a DataNode does not has free disk space, we can not get this
bad situation from anywhere, including DataNode log. At the same time,
under this situation, the hdfs writing operation will fail and return error
msg as below. However, from the error msg, user could not know the root
cause is that the only datanode runs out of disk space, and he also could
not get any useful hint in datanode log. So I believe it will be better if
we could add a regular check in DataNode on free disk space, and it will
add WARNING or ERROR msg in datanode log if that datanode runs out of
space. What's your opinion?

Error Msg:
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
/user/hadoop/PiEstimator_TMP_3_141592654/in/part0 could only be replicated
to 0 nodes instead of minReplication (=1).  There are 1 datanode(s) running
and no node(s) are excluded in this operation.
        at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1441)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2702)
        at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
        at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)


Thanks!

--047d7b66fbab0b70200505d1db5d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>Hi Experts and Developers,<br><br></div>At prese=
nt, if a DataNode does not has free disk space, we can not get this bad sit=
uation from anywhere, including DataNode log. At the same time, under this =
situation, the hdfs writing operation will fail and return error msg as bel=
ow. However, from the error msg, user could not know the root cause is that=
 the only datanode runs out of disk space, and he also could not get any us=
eful hint in datanode log. So I believe it will be better if we could add a=
 regular check in DataNode on free disk space, and it will add WARNING or E=
RROR msg in datanode log if that datanode runs out of space. What&#39;s you=
r opinion?<br><br>Error Msg:<br>org.apache.hadoop.ipc.RemoteException(java.=
io.IOException): File /user/hadoop/PiEstimator_TMP_3_141592654/in/part0 cou=
ld only be replicated to 0 nodes instead of minReplication (=3D1).=C2=A0 Th=
ere are 1 datanode(s) running and no node(s) are excluded in this operation=
.<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.s=
erver.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1441)<br>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.server=
.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2702)<br>=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.server.namen=
ode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.protocolPB.ClientN=
amenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServer=
SideTranslatorPB.java:440)<br><br><br></div>Thanks!<br></div>

--047d7b66fbab0b70200505d1db5d--