kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Burkert <...@cloudera.com>
Subject Re: sparkContext won't stop when using spark-kudu
Date Tue, 15 Mar 2016 01:56:40 GMT
Hi Darren,

I think the thread dump would be helpful.  We have a very similar test in
the repository, and we haven't had any problems with that.  What
environment are you running the job in?

- Dan

On Mon, Mar 14, 2016 at 8:20 AM, Darren Hoo <darren.hoo@gmail.com> wrote:

> I use sqlContext to register the kudu table
>
> sqlContext.read
>   .format("org.kududb.spark")
>   .options(Map("kudu.table" -> table, "kudu.master" -> kuduMaster))
>   .load()
>   .registerTempTable(table)
>
> then do something query and processing
>
>      sqlContext.sql("...")
>
>
> but after sc.stop() is called, the spark driver never exit:
>
> 16/03/14 22:54:51 INFO DAGScheduler: Stopping DAGScheduler
> 16/03/14 22:54:51 INFO YarnClientSchedulerBackend: Shutting down all
> executors
> 16/03/14 22:54:51 INFO YarnClientSchedulerBackend: Interrupting monitor
> thread
> 16/03/14 22:54:51 INFO YarnClientSchedulerBackend: Asking each
> executor to shut down
> 16/03/14 22:54:51 INFO YarnClientSchedulerBackend: Stopped
> 16/03/14 22:54:51 INFO MapOutputTrackerMasterEndpoint:
> MapOutputTrackerMasterEndpoint stopped!
> 16/03/14 22:54:51 INFO MemoryStore: MemoryStore cleared
> 16/03/14 22:54:51 INFO BlockManager: BlockManager stopped
> 16/03/14 22:54:51 INFO BlockManagerMaster: BlockManagerMaster stopped
> 16/03/14 22:54:51 INFO SparkContext: Successfully stopped SparkContext
> 16/03/14 22:54:51 INFO
> OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
> OutputCommitCoordinator stopped!
> 16/03/14 22:54:51 INFO RemoteActorRefProvider$RemotingTerminator:
> Shutting down remote daemon.
> 16/03/14 22:54:51 INFO RemoteActorRefProvider$RemotingTerminator:
> Remote daemon shut down; proceeding with flushing remote transports.
> 16/03/14 22:54:51 INFO Remoting: Remoting shut down
> 16/03/14 22:54:51 INFO RemoteActorRefProvider$RemotingTerminator:
> Remoting shut down.
>
> then it stuck here forever
>
> PS: I use spark on yarn client mode, the problem only occurs when I
> use kudu-spark
>
> I have the thread dump about 7k after gzipped, I can post here if asked.
>

Mime
View raw message