spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yong Zhang <java8...@hotmail.com>
Subject Re: keep or remove sc.stop() coz of RpcEnv already stopped error
Date Mon, 13 Mar 2017 13:31:25 GMT
What version of Spark you are using?


Based on Spark-12967, it is fixed on Spark 2.0 and later. If you are using Spark 1.x, you
can ignore this Warning. It shouldn't affect any functions.


Yong

________________________________
From: nancy henry <nancyhenry6542@gmail.com>
Sent: Monday, March 13, 2017 7:08 AM
To: user @spark
Subject: keep or remove sc.stop() coz of RpcEnv already stopped error

 Hi Team,

getting this error if we put sc.stop() in application..

can we remove it from application but i read if yu dont explicitly stop using sc.stop the
yarn application will not get registered in history service.. SO what to do ?

 WARN Dispatcher: Message RemoteProcessDisconnected dropped.
java.lang.IllegalStateException: RpcEnv already stopped.
        at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:159)
        at org.apache.spark.rpc.netty.Dispatcher.postToAll(Dispatcher.scala:109)
        at org.apache.spark.rpc.netty.NettyRpcHandler.connectionTerminated(NettyRpcEnv.scala:630)
        at org.apache.spark.network.server.TransportRequestHandler.channelUnregistered(TransportRequestHandler.java:94)



Mime
View raw message