hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject Re: issue about write append into hdfs "ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: ch12:50010:DataXceiver error processing READ_BLOCK operation "
Date Fri, 21 Feb 2014 05:51:02 GMT
one more question is if i need add the value of data node xceiver
need i add it to my NN config file?



On Fri, Feb 21, 2014 at 12:25 PM, Anurag Tangri <anurag_tangri@yahoo.com>wrote:

>  Did you check your unix open file limit and data node xceiver value ?
>
> Is it too low for the number of blocks/data in your cluster ?
>
> Thanks,
> Anurag Tangri
>
> On Feb 20, 2014, at 6:57 PM, ch huang <justlooks@gmail.com> wrote:
>
>   hi,maillist:
>           i see the following info in my hdfs log ,and the block belong to
> the file which write by scribe ,i do not know why
> is there any limit in hdfs system ?
>
> 2014-02-21 10:33:30,235 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: opReadBlock
> BP-1043055049-192.168.11.11-1382442676609:blk_-8536558734938003208_3823240
> received exc
> eption java.io.IOException: Replica gen stamp < block genstamp,
> block=BP-1043055049-192.168.11.11-1382442676609:blk_-8536558734938003208_3823240,
> replica=ReplicaWaitingToBeRecov
> ered, blk_-8536558734938003208_3820986, RWR
>   getNumBytes()     = 35840
>   getBytesOnDisk()  = 35840
>   getVisibleLength()= -1
>   getVolume()       = /data/4/dn/current
>   getBlockFile()    =
> /data/4/dn/current/BP-1043055049-192.168.11.11-1382442676609/current/rbw/blk_-8536558734938003208
>   unlinked=false
> 2014-02-21 10:33:30,235 WARN
> org.apache.hadoop.hdfs.server.datanode.DataNode:
> DatanodeRegistration(192.168.11.12,
> storageID=DS-754202132-192.168.11.12-50010-1382443087835, infoP
> ort=50075, ipcPort=50020,
> storageInfo=lv=-40;cid=CID-0e777b8c-19f3-44a1-8af1-916877f2506c;nsid=2086828354;c=0):Got
> exception while serving BP-1043055049-192.168.11.11-1382442676
> 609:blk_-8536558734938003208_3823240 to /192.168.11.15:56564
> java.io.IOException: Replica gen stamp < block genstamp,
> block=BP-1043055049-192.168.11.11-1382442676609:blk_-8536558734938003208_3823240,
> replica=ReplicaWaitingToBeRecovered, b
> lk_-8536558734938003208_3820986, RWR
>   getNumBytes()     = 35840
>   getBytesOnDisk()  = 35840
>   getVisibleLength()= -1
>   getVolume()       = /data/4/dn/current
>   getBlockFile()    =
> /data/4/dn/current/BP-1043055049-192.168.11.11-1382442676609/current/rbw/blk_-8536558734938003208
>   unlinked=false
>         at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:205)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:326)
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:92)
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:64)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-02-21 10:33:30,236 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: ch12:50010:DataXceiver
> error processing READ_BLOCK operation  src: /192.168.11.15:56564 dest: /
> 192.168.11.12:50010
> java.io.IOException: Replica gen stamp < block genstamp,
> block=BP-1043055049-192.168.11.11-1382442676609:blk_-8536558734938003208_3823240,
> replica=ReplicaWaitingToBeRecovered, blk_-8536558734938003208_3820986, RWR
>   getNumBytes()     = 35840
>   getBytesOnDisk()  = 35840
>   getVisibleLength()= -1
>   getVolume()       = /data/4/dn/current
>   getBlockFile()    =
> /data/4/dn/current/BP-1043055049-192.168.11.11-1382442676609/current/rbw/blk_-8536558734938003208
>   unlinked=false
>         at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:205)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:326)
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:92)
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:64)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)
>         at java.lang.Thread.run(Thread.java:744)
>
>

Mime
View raw message