spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From RodrigoB <>
Subject Re: broadcast variable get cleaned by ContextCleaner unexpectedly ?
Date Tue, 23 Sep 2014 11:37:08 GMT
Could you be using by any chance the getOrCreate for the StreamingContext

I've seen this happen when I tried to first create the Spark context, then
create the broadcast variables, and then recreate the StreamingContext from
the checkpoint directory. So the worker process cannot find the broadcast
variables that were used in the saved StreamingContext.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message