spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Rodríguez Hortalá <>
Subject Stopping StreamingContext before receiver has started
Date Mon, 13 Jul 2015 10:35:01 GMT

I have noticed that when StreamingContext.stop is called when no receiver
has started yet, then the context is not really stopped. Watching the logs
it looks like a stop signal is sent to 0 receivers, because the receivers
have not started yet, and then the receivers are started and the context
keeps working

15/07/13 12:24:52 INFO ReceiverTracker: Sent stop signal to all 0 receivers
15/07/13 12:24:52 INFO RecurringTimer: Started timer for BlockGenerator at
time 1436783092600
15/07/13 12:24:52 INFO BlockGenerator: Started BlockGenerator
15/07/13 12:24:52 INFO BlockGenerator: Started block pushing thread
15/07/13 12:24:52 INFO ReceiverSupervisorImpl: Starting receiver
15/07/13 12:24:52 INFO ActorReceiver: Supervision tree for receivers
initialized at:akka://sparkDriver/user/Supervisor0
15/07/13 12:24:52 INFO ActorReceiver: Started receiver worker
15/07/13 12:24:52 INFO ReceiverSupervisorImpl: Called receiver onStart
15/07/13 12:24:52 INFO ProxyReceiverActor: Starting
15/07/13 12:24:52 INFO ReceiverTracker: Registered receiver for stream 0

My workaround is using a streaming listener to wait for onReceiverStarted,
so I don't call stop before that event has occurred . This is a minor issue
but I think it is a surprising behaviour that maybe should be documented



View raw message