hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy <brahmared...@huawei.com>
Subject RE: could only be replicated to 0 nodes, instead of 1
Date Fri, 15 Jul 2011 08:54:30 GMT
Hi,

By seeing this exception(could only be replicated to 0 nodes, instead of 1)
,datanode is not available to Name Node..

This are the following cases Data Node may not available to Name Node

1)Data Node disk is Full

2)Data Node is Busy with block report and block scanning

3)If Block Size is Negative value(dfs.block.size in hdfs-site.xml)

4)while write in progress primary datanode goes down(Any n/w fluctations b/w
Name Node and Data Node Machines)

5)when Ever we append any partial chunk and call sync for subsequent partial
chunk appends client should store the previous data in buffer.

For example after appending "a" I have called sync and when I am trying the
to append the buffer should have "ab"

And Server side when the chunk is not multiple of 512 then it will try to do
Crc comparison for the data present in block file as well as crc present in
metafile. But while constructing crc for the data present in block it is
always comparing till the initial Offeset

Or For more analysis Please the data node logs

Warm Regards 

Brahma Reddy

****************************************************************************
***********
This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!
-----Original Message-----
From: Thomas Anderson [mailto:t.dt.aanderson@gmail.com] 
Sent: Friday, July 15, 2011 9:09 AM
To: hdfs-user@hadoop.apache.org
Subject: could only be replicated to 0 nodes, instead of 1

I have fresh  hadoop 0.20.2 installed on virtualbox 4.0.8 with jdk
1.6.0_26. The problem is when trying to put a file to hdfs, it throws
error `org.apache.hadoop.ipc.RemoteException: java.io.IOException:
File /path/to/file could only be replicated to 0 nodes, instead of 1';
however, there is no problem to create a folder, as the command ls
print the result

Found 1 items
drwxr-xr-x   - user supergroup          0 2011-07-15 11:09 /user/user/test

I also try with flushing firewall (remove all iptables restriction),
but the error message is still thrown out when uploading (hadoop fs
-put /tmp/x test) a file from local fs.

The name node log shows

2011-07-15 10:42:43,491 INFO org.apache.hadoop.hdfs.StateChange:
BLOCK* NameSystem.registerDatanode: node registration from
aaa.bbb.ccc.ddd.22:50010 storage DS-929017105-aaa.bbb.ccc.22-50010-13
10697763488
2011-07-15 10:42:43,495 INFO org.apache.hadoop.net.NetworkTopology:
Adding a new node: /default-rack/aaa.bbb.ccc.22:50010
2011-07-15 10:42:44,169 INFO org.apache.hadoop.hdfs.StateChange:
BLOCK* NameSystem.registerDatanode: node registration from
aaa.bbb.ccc.35:50010 storage DS-884574392-aaa.bbb.ccc.35-50010-13
10697764164
2011-07-15 10:42:44,170 INFO org.apache.hadoop.net.NetworkTopology:
Adding a new node: /default-rack/aaa.bbb.ccc.35:50010
2011-07-15 10:42:44,507 INFO org.apache.hadoop.hdfs.StateChange:
BLOCK* NameSystem.registerDatanode: node registration from
aaa.bbb.ccc.ddd.11:50010 storage DS-1537583073-aaa.bbb.ccc.11-50010-1
310697764488
2011-07-15 10:42:44,507 INFO org.apache.hadoop.net.NetworkTopology:
Adding a new node: /default-rack/aaa.bbb.ccc.11:50010
2011-07-15 10:42:45,796 INFO org.apache.hadoop.hdfs.StateChange:
BLOCK* NameSystem.registerDatanode: node registration from
140.127.220.25:50010 storage DS-1500589162-aaa.bbb.ccc.25-50010-1
310697765386
2011-07-15 10:42:45,797 INFO org.apache.hadoop.net.NetworkTopology:
Adding a new node: /default-rack/aaa.bbb.ccc.25:50010

And all datanodes have similar message as below:

2011-07-15 10:42:46,562 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: using
BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
2011-07-15 10:42:47,163 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0
blocks got processed in 3 msecs
2011-07-15 10:42:47,187 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic
block scanner.
2011-07-15 11:19:42,931 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0
blocks got processed in 1 msecs

Command `hadoop fsck /`  displays

Status: HEALTHY
 Total size:	0 B
 Total dirs:	3
 Total files:	0 (Files currently being written: 1)
 Total blocks (validated):	0
 Minimally replicated blocks:	0
 Over-replicated blocks:	0
 Under-replicated blocks:	0
 Mis-replicated blocks:		0
 Default replication factor:	3
 Average block replication:	0.0
 Corrupt blocks:		0
 Missing replicas:		0
 Number of data-nodes:		4

The setting in conf include:

- Master node:
core-site.xml
  <property>
    <name>fs.default.name</name>
    <value>hdfs://lab01:9000/</value>
  </property>

hdfs-site.xml
  <property>
    <name>dfs.replication</name>
    <value>3</value>
  </property>

-Slave nodes:
core-site.xml
  <property>
    <name>fs.default.name</name>
    <value>hdfs://lab01:9000/</value>
  </property>

hdfs-site.xml
 <property>
    <name>dfs.replication</name>
    <value>3</value>
  </property>

Do I missing any configuration? Or any place that I can check?

Thanks.


Mime
View raw message