Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of dontariq@gmail.com designates
 209.85.219.43 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <a8a2c65cc57910805a8dde26414b5ef2@ufl.edu>
References: <a5b41194211b4c189edc4ee07ccc0b0f@ufl.edu>
 <CAF4PTbx5dH74n8hYg56mShqu19HP6gOVSgaduoFNit+Hd4Lweg@mail.gmail.com>
 <2e03060e43aeb4ed462d36ecd1c7a9ef@ufl.edu>
 <a8a2c65cc57910805a8dde26414b5ef2@ufl.edu>
From: Mohammad Tariq <dontariq@gmail.com>
Date: Tue, 8 Oct 2013 23:51:30 +0530
Message-ID: 
 <CAMVC6RMunxy7nqNpu7oWysgzxP1H9OXXRmXgxqxcUXSEU+Bvcw@mail.gmail.com>
Subject: Re: Error putting files in the HDFS
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7bf18754e22ccf04e83ed81a

--047d7bf18754e22ccf04e83ed81a
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

You don't have any more space left in your HDFS. Delete some old data or
add additional storage.

Warm Regards,
Tariq
cloudfront.blogspot.com


On Tue, Oct 8, 2013 at 11:47 PM, Basu,Indrashish <indrashish@ufl.edu> wrote=
:

>
>
> Hi ,
>
> Just to update on this, I have deleted all the old logs and files from th=
e
> /tmp and /app/hadoop directory, and restarted all the nodes, I have now 1
> datanode available as per the below information :
>
> Configured Capacity: 3665985536 (3.41 GB)
> Present Capacity: 24576 (24 KB)
>
> DFS Remaining: 0 (0 KB)
> DFS Used: 24576 (24 KB)
> DFS Used%: 100%
>
> Under replicated blocks: 0
> Blocks with corrupt replicas: 0
> Missing blocks: 0
>
> ------------------------------**-------------------
> Datanodes available: 1 (1 total, 0 dead)
>
> Name: 10.227.56.195:50010
> Decommission Status : Normal
> Configured Capacity: 3665985536 (3.41 GB)
> DFS Used: 24576 (24 KB)
> Non DFS Used: 3665960960 (3.41 GB)
> DFS Remaining: 0(0 KB)
> DFS Used%: 0%
> DFS Remaining%: 0%
> Last contact: Tue Oct 08 11:12:19 PDT 2013
>
>
> However when I tried putting the files back in HDFS, I am getting the sam=
e
> error as stated earlier. Do I need to clear some space for the HDFS ?
>
> Regards,
> Indrashish
>
>
>
> On Tue, 08 Oct 2013 14:01:19 -0400, Basu,Indrashish wrote:
>
>> Hi Jitendra,
>>
>> This is what I am getting in the datanode logs :
>>
>> 2013-10-07 11:27:41,960 INFO
>> org.apache.hadoop.hdfs.server.**common.Storage: Storage directory
>> /app/hadoop/tmp/dfs/data is not formatted.
>> 2013-10-07 11:27:41,961 INFO
>> org.apache.hadoop.hdfs.server.**common.Storage: Formatting ...
>> 2013-10-07 11:27:42,094 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: Registered
>> FSDatasetStatusMBean
>> 2013-10-07 11:27:42,099 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: Opened info server at
>> 50010
>> 2013-10-07 11:27:42,107 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: Balancing bandwith is
>> 1048576 bytes/s
>> 2013-10-07 11:27:42,369 INFO org.mortbay.log: Logging to
>> org.slf4j.impl.**Log4jLoggerAdapter(org.**mortbay.log) via
>> org.mortbay.log.Slf4jLog
>> 2013-10-07 11:27:42,632 INFO org.apache.hadoop.http.**HttpServer: Port
>> returned by webServer.getConnectors()[0].**getLocalPort() before open()
>> is -1. Opening the listener on 50075
>> 2013-10-07 11:27:42,633 INFO org.apache.hadoop.http.**HttpServer:
>> listener.getLocalPort() returned 50075
>> webServer.getConnectors()[0].**getLocalPort() returned 50075
>> 2013-10-07 11:27:42,634 INFO org.apache.hadoop.http.**HttpServer: Jetty
>> bound to port 50075
>> 2013-10-07 11:27:42,634 INFO org.mortbay.log: jetty-6.1.14
>> 2013-10-07 11:31:29,821 INFO org.mortbay.log: Started
>> SelectChannelConnector@0.0.0.**0:50075<http://SelectChannelConnector@0.0=
.0.0:50075>
>> 2013-10-07 11:31:29,843 INFO
>> org.apache.hadoop.metrics.jvm.**JvmMetrics: Initializing JVM Metrics
>> with processName=3DDataNode, sessionId=3Dnull
>> 2013-10-07 11:31:29,912 INFO
>> org.apache.hadoop.ipc.metrics.**RpcMetrics: Initializing RPC Metrics
>> with hostName=3DDataNode, port=3D50020
>> 2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server
>> Responder: starting
>> 2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server
>> listener on 50020: starting
>> 2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 0 on 50020: starting
>> 2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 1 on 50020: starting
>> 2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 2 on 50020: starting
>> 2013-10-07 11:31:29,934 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: dnRegistration =3D
>> DatanodeRegistration(tegra-**ubuntu:50010, storageID=3D, infoPort=3D5007=
5,
>> ipcPort=3D50020)
>> 2013-10-07 11:31:29,971 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: New storage id
>> DS-1027334635-127.0.1.1-50010-**1381170689938 is assigned to data-node
>> 10.227.56.195:50010
>> 2013-10-07 11:31:29,973 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode:
>> DatanodeRegistration(10.227.**56.195:50010 <http://10.227.56.195:50010>,
>> storageID=3DDS-1027334635-127.0.**1.1-50010-1381170689938, infoPort=3D50=
075,
>> ipcPort=3D50020)In DataNode.run, data =3D FSDataset
>> {dirpath=3D'/app/hadoop/tmp/dfs/**data/current'}
>> 2013-10-07 11:31:29,974 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: using
>> BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
>> 2013-10-07 11:31:30,032 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: BlockReport of 0
>> blocks got processed in 19 msecs
>> 2013-10-07 11:31:30,035 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: Starting Periodic
>> block scanner.
>> 2013-10-07 11:41:42,222 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: BlockReport of 0
>> blocks got processed in 20 msecs
>> 2013-10-07 12:41:43,482 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: BlockReport of 0
>> blocks got processed in 22 msecs
>> 2013-10-07 13:41:44,755 INFO
>> org.apache.hadoop.hdfs.server.**datanode.DataNode: BlockReport of 0
>> blocks got processed in 13 msecs
>>
>>
>> I restarted the datanode and made sure that it is up and running
>> (typed jps command).
>>
>> Regards,
>> Indrashish
>>
>> On Tue, 8 Oct 2013 23:25:25 +0530, Jitendra Yadav wrote:
>>
>>> As per your dfs report, available DataNodes  count is ZERO  in you
>>> cluster.
>>>
>>> Please check your data node logs.
>>>
>>> Regards
>>> Jitendra
>>>
>>> On 10/8/13, Basu,Indrashish <indrashish@ufl.edu> wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>> My name is Indrashish Basu and I am a Masters student in the Departmen=
t
>>>> of Electrical and Computer Engineering.
>>>>
>>>> Currently I am doing my research project on Hadoop implementation on
>>>> ARM processor and facing an issue while trying to run a sample Hadoop
>>>> source code on the same. Every time I am trying to put some files in t=
he
>>>> HDFS, I am getting the below error.
>>>>
>>>>
>>>> 13/10/07 11:31:29 WARN hdfs.DFSClient: DataStreamer Exception:
>>>> org.apache.hadoop.ipc.**RemoteException: java.io.IOException: File
>>>> /user/root/bin/cpu-kmeans2D could only be replicated to 0 nodes, inste=
ad
>>>> of 1
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.server.**namenode.FSNamesystem.**
>>>> getAdditionalBlock(**FSNamesystem.java:1267)
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.server.**namenode.NameNode.addBlock(**
>>>> NameNode.java:422)
>>>> at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
>>>> at
>>>>
>>>> sun.reflect.**NativeMethodAccessorImpl.**invoke(**
>>>> NativeMethodAccessorImpl.java:**57)
>>>> at
>>>>
>>>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>>>> DelegatingMethodAccessorImpl.**java:43)
>>>> at java.lang.reflect.Method.**invoke(Method.java:606)
>>>> at org.apache.hadoop.ipc.RPC$**Server.call(RPC.java:508)
>>>> at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:959)
>>>> at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:955)
>>>> at java.security.**AccessController.doPrivileged(**Native Method)
>>>> at javax.security.auth.Subject.**doAs(Subject.java:415)
>>>> at org.apache.hadoop.ipc.Server$**Handler.run(Server.java:953)
>>>>
>>>> at org.apache.hadoop.ipc.Client.**call(Client.java:739)
>>>> at org.apache.hadoop.ipc.RPC$**Invoker.invoke(RPC.java:220)
>>>> at com.sun.proxy.$Proxy0.**addBlock(Unknown Source)
>>>> at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
>>>> at
>>>>
>>>> sun.reflect.**NativeMethodAccessorImpl.**invoke(**
>>>> NativeMethodAccessorImpl.java:**57)
>>>> at
>>>>
>>>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>>>> DelegatingMethodAccessorImpl.**java:43)
>>>> at java.lang.reflect.Method.**invoke(Method.java:606)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.**RetryInvocationHandler.**invokeMethod(**
>>>> RetryInvocationHandler.java:**82)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.**RetryInvocationHandler.invoke(**
>>>> RetryInvocationHandler.java:**59)
>>>> at com.sun.proxy.$Proxy0.**addBlock(Unknown Source)
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.**DFSClient$DFSOutputStream.**
>>>> locateFollowingBlock(**DFSClient.java:2904)
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.**DFSClient$DFSOutputStream.**
>>>> nextBlockOutputStream(**DFSClient.java:2786)
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.**DFSClient$DFSOutputStream.**
>>>> access$2000(DFSClient.java:**2076)
>>>> at
>>>>
>>>> org.apache.hadoop.hdfs.**DFSClient$DFSOutputStream$**
>>>> DataStreamer.run(DFSClient.**java:2262)
>>>>
>>>> 13/10/07 11:31:29 WARN hdfs.DFSClient: Error Recovery for block null
>>>> bad datanode[0] nodes =3D=3D null
>>>> 13/10/07 11:31:29 WARN hdfs.DFSClient: Could not get block locations.
>>>> Source file "/user/root/bin/cpu-kmeans2D" - Aborting...
>>>> put: java.io.IOException: File /user/root/bin/cpu-kmeans2D could only
>>>> be replicated to 0 nodes, instead of 1
>>>>
>>>>
>>>> I tried replicating the namenode and datanode by deleting all the old
>>>> logs on the master and the slave nodes as well as the folders under
>>>> /app/hadoop/, after which I formatted the namenode and started the
>>>> process again (bin/start-all.sh), but still no luck with the same.
>>>>
>>>> I tried generating the admin report(pasted below) after doing the
>>>> restart, it seems the data node is not getting started.
>>>>
>>>> ------------------------------**-------------------
>>>> Datanodes available: 0 (0 total, 0 dead)
>>>>
>>>> root@tegra-ubuntu:~/hadoop-**gpu-master/hadoop-gpu-0.20.1# bin/hadoop
>>>> dfsadmin -report
>>>> Configured Capacity: 0 (0 KB)
>>>> Present Capacity: 0 (0 KB)
>>>> DFS Remaining: 0 (0 KB)
>>>> DFS Used: 0 (0 KB)
>>>> DFS Used%: =EF=BF=BD%
>>>> Under replicated blocks: 0
>>>> Blocks with corrupt replicas: 0
>>>> Missing blocks: 0
>>>>
>>>> ------------------------------**-------------------
>>>> Datanodes available: 0 (0 total, 0 dead)
>>>>
>>>>
>>>> I have tried the following methods to debug the process :
>>>>
>>>> 1) I logged in to the HADOOP home directory and removed all the old
>>>> logs (rm -rf logs/*)
>>>>
>>>> 2) Next I deleted the contents of the directory on all my slave and
>>>> master nodes (rm -rf /app/hadoop/*)
>>>>
>>>> 3) I formatted the namenode (bin/hadoop namenode -format)
>>>>
>>>> 4) I started all the processes - first the namenode, datanode and then
>>>> the map - reduce. I typed jps on the terminal to ensure that all the
>>>> processes (Namenode, Datanode, JobTracker, Task Tracker) are up and
>>>> running.
>>>>
>>>> 5) Now doing this, I recreated the directories in the dfs.
>>>>
>>>> However still no luck with the process.
>>>>
>>>>
>>>> Can you kindly assist regarding this ? I am new to Hadoop and I am
>>>> having no idea as how I can proceed with this.
>>>>
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> --
>>>> Indrashish Basu
>>>> Graduate Student
>>>> Department of Electrical and Computer Engineering
>>>> University of Florida
>>>>
>>>>
> --
> Indrashish Basu
> Graduate Student
> Department of Electrical and Computer Engineering
> University of Florida
>

--047d7bf18754e22ccf04e83ed81a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">You don&#39;t have any more space left in your HDFS. Delet=
e some old data or add additional storage.</div><div class=3D"gmail_extra">=
<br clear=3D"all"><div><div dir=3D"ltr">Warm Regards,<div>Tariq</div><div><=
a href=3D"http://cloudfront.blogspot.com" target=3D"_blank">cloudfront.blog=
spot.com</a><br>

</div></div></div>
<br><br><div class=3D"gmail_quote">On Tue, Oct 8, 2013 at 11:47 PM, Basu,In=
drashish <span dir=3D"ltr">&lt;<a href=3D"mailto:indrashish@ufl.edu" target=
=3D"_blank">indrashish@ufl.edu</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">

<br>
<br>
Hi ,<br>
<br>
Just to update on this, I have deleted all the old logs and files from the =
/tmp and /app/hadoop directory, and restarted all the nodes, I have now 1 d=
atanode available as per the below information :<br>
<br>
Configured Capacity: 3665985536 (3.41 GB)<br>
Present Capacity: 24576 (24 KB)<div class=3D"im"><br>
DFS Remaining: 0 (0 KB)<br></div>
DFS Used: 24576 (24 KB)<br>
DFS Used%: 100%<div class=3D"im"><br>
Under replicated blocks: 0<br>
Blocks with corrupt replicas: 0<br>
Missing blocks: 0<br>
<br>
------------------------------<u></u>-------------------<br></div>
Datanodes available: 1 (1 total, 0 dead)<br>
<br>
Name: <a href=3D"http://10.227.56.195:50010" target=3D"_blank">10.227.56.19=
5:50010</a><br>
Decommission Status : Normal<br>
Configured Capacity: 3665985536 (3.41 GB)<br>
DFS Used: 24576 (24 KB)<br>
Non DFS Used: 3665960960 (3.41 GB)<br>
DFS Remaining: 0(0 KB)<br>
DFS Used%: 0%<br>
DFS Remaining%: 0%<br>
Last contact: Tue Oct 08 11:12:19 PDT 2013<br>
<br>
<br>
However when I tried putting the files back in HDFS, I am getting the same =
error as stated earlier. Do I need to clear some space for the HDFS ?<br>
<br>
Regards,<br>
Indrashish<div class=3D"HOEnZb"><div class=3D"h5"><br>
<br>
<br>
On Tue, 08 Oct 2013 14:01:19 -0400, Basu,Indrashish wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
Hi Jitendra,<br>
<br>
This is what I am getting in the datanode logs :<br>
<br>
2013-10-07 11:27:41,960 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>common.Storage: Storage directory<br>
/app/hadoop/tmp/dfs/data is not formatted.<br>
2013-10-07 11:27:41,961 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>common.Storage: Formatting ...<br>
2013-10-07 11:27:42,094 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: Registered<br>
FSDatasetStatusMBean<br>
2013-10-07 11:27:42,099 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: Opened info server =
at<br>
50010<br>
2013-10-07 11:27:42,107 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: Balancing bandwith =
is<br>
1048576 bytes/s<br>
2013-10-07 11:27:42,369 INFO org.mortbay.log: Logging to<br>
org.slf4j.impl.<u></u>Log4jLoggerAdapter(org.<u></u>mortbay.log) via<br>
org.mortbay.log.Slf4jLog<br>
2013-10-07 11:27:42,632 INFO org.apache.hadoop.http.<u></u>HttpServer: Port=
<br>
returned by webServer.getConnectors()[0].<u></u>getLocalPort() before open(=
)<br>
is -1. Opening the listener on 50075<br>
2013-10-07 11:27:42,633 INFO org.apache.hadoop.http.<u></u>HttpServer:<br>
listener.getLocalPort() returned 50075<br>
webServer.getConnectors()[0].<u></u>getLocalPort() returned 50075<br>
2013-10-07 11:27:42,634 INFO org.apache.hadoop.http.<u></u>HttpServer: Jett=
y<br>
bound to port 50075<br>
2013-10-07 11:27:42,634 INFO org.mortbay.log: jetty-6.1.14<br>
2013-10-07 11:31:29,821 INFO org.mortbay.log: Started<br>
<a href=3D"http://SelectChannelConnector@0.0.0.0:50075" target=3D"_blank">S=
electChannelConnector@0.0.0.<u></u>0:50075</a><br>
2013-10-07 11:31:29,843 INFO<br>
org.apache.hadoop.metrics.jvm.<u></u>JvmMetrics: Initializing JVM Metrics<b=
r>
with processName=3DDataNode, sessionId=3Dnull<br>
2013-10-07 11:31:29,912 INFO<br>
org.apache.hadoop.ipc.metrics.<u></u>RpcMetrics: Initializing RPC Metrics<b=
r>
with hostName=3DDataNode, port=3D50020<br>
2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server<br>
Responder: starting<br>
2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server<br>
listener on 50020: starting<br>
2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server<br>
handler 0 on 50020: starting<br>
2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server<br>
handler 1 on 50020: starting<br>
2013-10-07 11:31:29,933 INFO org.apache.hadoop.ipc.Server: IPC Server<br>
handler 2 on 50020: starting<br>
2013-10-07 11:31:29,934 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: dnRegistration =3D<=
br>
DatanodeRegistration(tegra-<u></u>ubuntu:50010, storageID=3D, infoPort=3D50=
075,<br>
ipcPort=3D50020)<br>
2013-10-07 11:31:29,971 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: New storage id<br>
DS-1027334635-127.0.1.1-50010-<u></u>1381170689938 is assigned to data-node=
<br>
<a href=3D"http://10.227.56.195:50010" target=3D"_blank">10.227.56.195:5001=
0</a><br>
2013-10-07 11:31:29,973 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode:<br>
DatanodeRegistration(<a href=3D"http://10.227.56.195:50010" target=3D"_blan=
k">10.227.<u></u>56.195:50010</a>,<br>
storageID=3DDS-1027334635-127.0.<u></u>1.1-50010-1381170689938, infoPort=3D=
50075,<br>
ipcPort=3D50020)In DataNode.run, data =3D FSDataset<br>
{dirpath=3D&#39;/app/hadoop/tmp/dfs/<u></u>data/current&#39;}<br>
2013-10-07 11:31:29,974 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: using<br>
BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec<br>
2013-10-07 11:31:30,032 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: BlockReport of 0<br=
>
blocks got processed in 19 msecs<br>
2013-10-07 11:31:30,035 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: Starting Periodic<b=
r>
block scanner.<br>
2013-10-07 11:41:42,222 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: BlockReport of 0<br=
>
blocks got processed in 20 msecs<br>
2013-10-07 12:41:43,482 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: BlockReport of 0<br=
>
blocks got processed in 22 msecs<br>
2013-10-07 13:41:44,755 INFO<br>
org.apache.hadoop.hdfs.server.<u></u>datanode.DataNode: BlockReport of 0<br=
>
blocks got processed in 13 msecs<br>
<br>
<br>
I restarted the datanode and made sure that it is up and running<br>
(typed jps command).<br>
<br>
Regards,<br>
Indrashish<br>
<br>
On Tue, 8 Oct 2013 23:25:25 +0530, Jitendra Yadav wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
As per your dfs report, available DataNodes =C2=A0count is ZERO =C2=A0in yo=
u cluster.<br>
<br>
Please check your data node logs.<br>
<br>
Regards<br>
Jitendra<br>
<br>
On 10/8/13, Basu,Indrashish &lt;<a href=3D"mailto:indrashish@ufl.edu" targe=
t=3D"_blank">indrashish@ufl.edu</a>&gt; wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
Hello,<br>
<br>
My name is Indrashish Basu and I am a Masters student in the Department<br>
of Electrical and Computer Engineering.<br>
<br>
Currently I am doing my research project on Hadoop implementation on<br>
ARM processor and facing an issue while trying to run a sample Hadoop<br>
source code on the same. Every time I am trying to put some files in the<br=
>
HDFS, I am getting the below error.<br>
<br>
<br>
13/10/07 11:31:29 WARN hdfs.DFSClient: DataStreamer Exception:<br>
org.apache.hadoop.ipc.<u></u>RemoteException: java.io.IOException: File<br>
/user/root/bin/cpu-kmeans2D could only be replicated to 0 nodes, instead<br=
>
of 1<br>
at<br>
<br>
org.apache.hadoop.hdfs.server.<u></u>namenode.FSNamesystem.<u></u>getAdditi=
onalBlock(<u></u>FSNamesystem.java:1267)<br>
at<br>
<br>
org.apache.hadoop.hdfs.server.<u></u>namenode.NameNode.addBlock(<u></u>Name=
Node.java:422)<br>
at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke0(Native Method=
)<br>
at<br>
<br>
sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke(<u></u>NativeMeth=
odAccessorImpl.java:<u></u>57)<br>
at<br>
<br>
sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>Delega=
tingMethodAccessorImpl.<u></u>java:43)<br>
at java.lang.reflect.Method.<u></u>invoke(Method.java:606)<br>
at org.apache.hadoop.ipc.RPC$<u></u>Server.call(RPC.java:508)<br>
at org.apache.hadoop.ipc.Server$<u></u>Handler$1.run(Server.java:959)<br>
at org.apache.hadoop.ipc.Server$<u></u>Handler$1.run(Server.java:955)<br>
at java.security.<u></u>AccessController.doPrivileged(<u></u>Native Method)=
<br>
at javax.security.auth.Subject.<u></u>doAs(Subject.java:415)<br>
at org.apache.hadoop.ipc.Server$<u></u>Handler.run(Server.java:953)<br>
<br>
at org.apache.hadoop.ipc.Client.<u></u>call(Client.java:739)<br>
at org.apache.hadoop.ipc.RPC$<u></u>Invoker.invoke(RPC.java:220)<br>
at com.sun.proxy.$Proxy0.<u></u>addBlock(Unknown Source)<br>
at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke0(Native Method=
)<br>
at<br>
<br>
sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke(<u></u>NativeMeth=
odAccessorImpl.java:<u></u>57)<br>
at<br>
<br>
sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>Delega=
tingMethodAccessorImpl.<u></u>java:43)<br>
at java.lang.reflect.Method.<u></u>invoke(Method.java:606)<br>
at<br>
<br>
org.apache.hadoop.io.retry.<u></u>RetryInvocationHandler.<u></u>invokeMetho=
d(<u></u>RetryInvocationHandler.java:<u></u>82)<br>
at<br>
<br>
org.apache.hadoop.io.retry.<u></u>RetryInvocationHandler.invoke(<u></u>Retr=
yInvocationHandler.java:<u></u>59)<br>
at com.sun.proxy.$Proxy0.<u></u>addBlock(Unknown Source)<br>
at<br>
<br>
org.apache.hadoop.hdfs.<u></u>DFSClient$DFSOutputStream.<u></u>locateFollow=
ingBlock(<u></u>DFSClient.java:2904)<br>
at<br>
<br>
org.apache.hadoop.hdfs.<u></u>DFSClient$DFSOutputStream.<u></u>nextBlockOut=
putStream(<u></u>DFSClient.java:2786)<br>
at<br>
<br>
org.apache.hadoop.hdfs.<u></u>DFSClient$DFSOutputStream.<u></u>access$2000(=
DFSClient.java:<u></u>2076)<br>
at<br>
<br>
org.apache.hadoop.hdfs.<u></u>DFSClient$DFSOutputStream$<u></u>DataStreamer=
.run(DFSClient.<u></u>java:2262)<br>
<br>
13/10/07 11:31:29 WARN hdfs.DFSClient: Error Recovery for block null<br>
bad datanode[0] nodes =3D=3D null<br>
13/10/07 11:31:29 WARN hdfs.DFSClient: Could not get block locations.<br>
Source file &quot;/user/root/bin/cpu-kmeans2D&quot; - Aborting...<br>
put: java.io.IOException: File /user/root/bin/cpu-kmeans2D could only<br>
be replicated to 0 nodes, instead of 1<br>
<br>
<br>
I tried replicating the namenode and datanode by deleting all the old<br>
logs on the master and the slave nodes as well as the folders under<br>
/app/hadoop/, after which I formatted the namenode and started the<br>
process again (bin/start-all.sh), but still no luck with the same.<br>
<br>
I tried generating the admin report(pasted below) after doing the<br>
restart, it seems the data node is not getting started.<br>
<br>
------------------------------<u></u>-------------------<br>
Datanodes available: 0 (0 total, 0 dead)<br>
<br>
root@tegra-ubuntu:~/hadoop-<u></u>gpu-master/hadoop-gpu-0.20.1# bin/hadoop<=
br>
dfsadmin -report<br>
Configured Capacity: 0 (0 KB)<br>
Present Capacity: 0 (0 KB)<br>
DFS Remaining: 0 (0 KB)<br>
DFS Used: 0 (0 KB)<br>
DFS Used%: =EF=BF=BD%<br>
Under replicated blocks: 0<br>
Blocks with corrupt replicas: 0<br>
Missing blocks: 0<br>
<br>
------------------------------<u></u>-------------------<br>
Datanodes available: 0 (0 total, 0 dead)<br>
<br>
<br>
I have tried the following methods to debug the process :<br>
<br>
1) I logged in to the HADOOP home directory and removed all the old<br>
logs (rm -rf logs/*)<br>
<br>
2) Next I deleted the contents of the directory on all my slave and<br>
master nodes (rm -rf /app/hadoop/*)<br>
<br>
3) I formatted the namenode (bin/hadoop namenode -format)<br>
<br>
4) I started all the processes - first the namenode, datanode and then<br>
the map - reduce. I typed jps on the terminal to ensure that all the<br>
processes (Namenode, Datanode, JobTracker, Task Tracker) are up and<br>
running.<br>
<br>
5) Now doing this, I recreated the directories in the dfs.<br>
<br>
However still no luck with the process.<br>
<br>
<br>
Can you kindly assist regarding this ? I am new to Hadoop and I am<br>
having no idea as how I can proceed with this.<br>
<br>
<br>
<br>
<br>
Regards,<br>
<br>
--<br>
Indrashish Basu<br>
Graduate Student<br>
Department of Electrical and Computer Engineering<br>
University of Florida<br>
<br>
</blockquote></blockquote></blockquote>
<br>
-- <br>
Indrashish Basu<br>
Graduate Student<br>
Department of Electrical and Computer Engineering<br>
University of Florida<br>
</div></div></blockquote></div><br></div>

--047d7bf18754e22ccf04e83ed81a--