hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Kartha <karth...@gmail.com>
Subject DataNode logs have exceptions - DataXceiver error processing unknown operation
Date Thu, 25 Jun 2015 23:57:57 GMT
Hello,

I am using a Hadoop 2.7.1 build and noticed a constant flow of exceptions
every 60 seconds in the DataNode log files:

2015-06-25 13:02:36,292 ERROR datanode.DataNode (DataXceiver.java:run(278))
- bdavm063.svl.ibm.com:50010:DataXceiver error processing unknown
operation  src: /127.0.0.1:54415 dst: /127.0.0.1:50010
java.io.EOFException
        at java.io.DataInputStream.readShort(DataInputStream.java:315)
        at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58)
        at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:227)
        at java.lang.Thread.run(Thread.java:745)
2015-06-25 13:03:34,328 ERROR datanode.DataNode (DataXceiver.java:run(278))
- bdavm063.svl.ibm.com:50010:DataXceiver error processing unknown
operation  src: /127.0.0.1:54441 dst: /127.0.0.1:50010
java.io.EOFException
        at java.io.DataInputStream.readShort(DataInputStream.java:315)
        at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58)
        at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:227)
        at java.lang.Thread.run(Thread.java:745)
2015-06-25 13:05:36,081 ERROR datanode.DataNode (DataXceiver.java:run(278))
- bdavm063.svl.ibm.com:50010:DataXceiver error processing unknown
operation  src: /127.0.0.1:54477 dst: /127.0.0.1:50010
java.io.EOFException
        at java.io.DataInputStream.readShort(DataInputStream.java:315)
        at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58)
        at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:227)
        at java.lang.Thread.run(Thread.java:745)



I checked:
-  ulimit to be -  open files                      (-n) *32768*
- dfs.datanode.max.transfer.threads to be *16384*

While HDFS seem to work without issues, the logs are filled with these.

Any thoughts/ideas on resolving this is greatly appreciated.

Regards,
Rajesh

Mime
View raw message