hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanbo Liang <yanboha...@gmail.com>
Subject IOException in offerService
Date Tue, 13 Nov 2012 10:51:43 GMT
DataNode may received IOException in heartbeat sending process. When
running TestStartup, I got the following warn log:

2012-11-13 01:28:19,217 WARN  datanode.DataNode
(BPServiceActor.java:offerService(596)) - IOException in offerService
java.io.IOException: Failed on local exception: java.io.IOException:
Response is null.; Host Details : local host is: "orange/127.0.0.1";
destination host is: "orange":38705;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
        at org.apache.hadoop.ipc.Client.call(Client.java:1163)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
        at $Proxy14.sendHeartbeat(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:163)
        at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:82)
        at $Proxy14.sendHeartbeat(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:167)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:445)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:525)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:676)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Response is null.
        at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:884)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:812)

>From the log, I can infer that DN send heartbeat to NN and got response is
null.
But the workflow of process the response at DN just log it and without
other action.
I wonder which caused this result? The NameNode has no DatanodeCommand to
send for this DataNode, protocolbuffer related problem or other reason?
Is it make sense that the DN just log this Exception and without other
action?

Thanks,
Yanbo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message