spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From RodrigoB <rodrigo.boav...@aspect.com>
Subject Re: broadcast variable get cleaned by ContextCleaner unexpectedly ?
Date Tue, 23 Sep 2014 11:37:08 GMT
Could you be using by any chance the getOrCreate for the StreamingContext
creation?

I've seen this happen when I tried to first create the Spark context, then
create the broadcast variables, and then recreate the StreamingContext from
the checkpoint directory. So the worker process cannot find the broadcast
variables that were used in the saved StreamingContext.

tnks,
Rod



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-variable-get-cleaned-by-ContextCleaner-unexpectedly-tp10347p14884.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message