kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhen Zhang <zhqu...@gmail.com>
Subject RPC negotiation error
Date Fri, 01 Dec 2017 03:32:42 GMT
Hi all, I found some rpc negotiation errors on both master and client side.
I'm not sure what cause this, on both client and master server, I see no
network errors using ifconfig. Do you know what may cause this error occur?

I'm using Kudu1.3.0, and the security configuration is disabled:
--rpc_encryption=disabled
--rpc_authentication=disabled

The master side error:
W1201 09:16:05.300465 71450 negotiation.cc:290] Failed RPC negotiation.
Trace:
1201 09:15:50.300130 (+     0us) reactor.cc:373] Submitting negotiation
task for server connection from 10.68.11.135:42847
1201 09:15:50.300446 (+   316us) server_negotiation.cc:118] Beginning
negotiation
1201 09:15:50.300448 (+     2us) server_negotiation.cc:282] Waiting for
connection header
1201 09:16:05.300393 (+14999945us) negotiation.cc:281] Negotiation
complete: Timed out: Server connection negotiation failed: server
connection from 10.68.11.135:42847
Metrics:
{"negotiator.queue_time_us":237,"thread_start_us":213,"threads_started":1}


The client side error:
2017-12-01,09:16:46,063 ERROR org.apache.kudu.client.TabletClient: [Peer
master-sgp1-hadoop-prc-kudu01.awssgp:18600] Unexpected exception from
downstream on [id: 0x21836b16, /10.68.11.135:42847 =>
sgp1-hadoop-prc-kudu01.awssgp/10.68.1.73:18600]
java.io.IOException: Broken pipe
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
        at sun.nio.ch.IOUtil.write(IOUtil.java:51)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:146)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:84)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:725)
        at
org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71)
        at
org.apache.kudu.client.shaded.org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:704)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.Channels.write(Channels.java:671)
        at
org.apache.kudu.client.Negotiator.sendSaslMessage(Negotiator.java:218)
        at
org.apache.kudu.client.Negotiator.sendNegotiateMessage(Negotiator.java:211)
        at org.apache.kudu.client.Negotiator.sendHello(Negotiator.java:178)
        at
org.apache.kudu.client.TabletClient.channelConnected(TabletClient.java:586)
        at
org.apache.kudu.client.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:100)
        at
org.apache.kudu.client.TabletClient.handleUpstream(TabletClient.java:595)

Thanks,
Zhen

Mime
View raw message