Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 31 Aug 2015 11:39:44 -0700 (MST)
From: nasokan <anithin19@gmail.com>
To: user@spark.apache.org
Message-ID: <1441046384353-24523.post@n3.nabble.com>
Subject: Potential NPE while exiting spark-shell
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I'm currently using Spark 1.3.0 on yarn cluster deployed through CDH5.4. My
cluster does not have a 'default' queue, and launching 'spark-shell' submits
an yarn application that gets killed immediately because queue does not
exist. However, the spark-shell session is still in progress after throwing
a bunch of errors while creating sql context. Upon submitting an 'exit'
command, there appears to be a NPE from DiskBlockManager with the following
stack trace

ERROR Utils: Uncaught exception in thread delete Spark local dirs
java.lang.NullPointerException
	at
org.apache.spark.storage.DiskBlockManager.org$apache$spark$storage$DiskBlockManager$$doStop(DiskBlockManager.scala:161)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply$mcV$sp(DiskBlockManager.scala:141)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
	at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1617)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1.run(DiskBlockManager.scala:139)
Exception in thread "delete Spark local dirs" java.lang.NullPointerException
	at
org.apache.spark.storage.DiskBlockManager.org$apache$spark$storage$DiskBlockManager$$doStop(DiskBlockManager.scala:161)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply$mcV$sp(DiskBlockManager.scala:141)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
	at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1617)
	at
org.apache.spark.storage.DiskBlockManager$$anon$1.run(DiskBlockManager.scala:139)

I believe the problem appears to be surfacing from a shutdown hook that's
tries to cleanup local directories. In this specific case because the yarn
application was not submitted successfully, the block manager was not
registered; as a result it does not have a valid blockManagerId as seen here

https://github.com/apache/spark/blob/v1.3.0/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala#L161

Has anyone faced this issue before? Could this be a problem with the way
shutdown hook behaves currently?

Note: I referenced source from apache spark repo than cloudera.


--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Potential-NPE-while-exiting-spark-shell-tp24523.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org