spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kundan kumar <iitr.kun...@gmail.com>
Subject Executor still on the UI even if the worker is dead
Date Wed, 20 Apr 2016 16:26:56 GMT
Hi TD/Cody,

Why does it happen so in Spark Streaming that the executors are still shown
on the UI even when the worker is killed and not in the cluster.

This severely impacts my running jobs which takes too longer and the stages
failing with the exception

java.io.IOException: Failed to connect to --- (dead worker)

Is this a bug in Spark ??

Version is 1.4.0

This is entirely against the fault tolerance of the workers. Killing a
worker in a cluster of 5 impacts the entire job.

Thanks,
Kundan

Mime
View raw message