spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattias Aspholm (JIRA)" <>
Subject [jira] [Commented] (SPARK-14846) Driver process fails to terminate when graceful shutdown is used
Date Sat, 23 Apr 2016 20:19:13 GMT


Mattias Aspholm commented on SPARK-14846:

Yes, you're right of course. Sorry about that. I'm still having problems with the driver not
closing down in graceful (even though there's no work left), but I realise now my initial
conclusions was bad, the reason why it hangs in awaitTermination is that the termination condition
is not signaled. I need to find out why that happens.

Ok for me to close this bug as invalid. I'll file another one if it turns out to be some bug
after all.

> Driver process fails to terminate when graceful shutdown is used
> ----------------------------------------------------------------
>                 Key: SPARK-14846
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.6.1
>            Reporter: Mattias Aspholm
> During shutdown, the job scheduler in Streaming (JobScheduler.stop) spends some time
waiting for all queued work to complete. If graceful shutdown is used, the time is 1 hour,
for non-graceful shutdown it's 2 seconds.
> The wait is implemented using the ThreadPoolExecutor.awaitTermination method in java.util.concurrent.
The problem is that instead of looping over the method for the desired period of time, the
wait period is passed in as the timeout parameter to awaitTermination. 
> The result is that if the termination condition is false the first time, the method will
sleep for the timeout period before trying again. In the case of graceful shutdown this means
at least an hour's wait before the condition is checked again, even though all work is completed
in just a few seconds. The driver process will continue to live during this time.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message