spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Orka (JIRA)" <>
Subject [jira] [Commented] (SPARK-17707) Web UI prevents spark-submit application to be finished
Date Thu, 06 Oct 2016 11:31:20 GMT


Nick Orka commented on SPARK-17707:

Calling stop() would be a good idea if there were not getOrCreate(). Spark context is reusable.
If you start two simultaneous threads when are you supposed to stop() the context explicitly?

Is it better to fix the issue for backward compatibility or make context non-reusable?

> Web UI prevents spark-submit application to be finished
> -------------------------------------------------------
>                 Key: SPARK-17707
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 2.0.1
>            Reporter: Nick Orka
> Here are re-production steps:
> 1. create any scala spark application which will work long enough to open the application
details in Web UI
> 2. run spark-submit command for standalone cluster, like: --master spark:\\localhost:7077
> 3. open running application details in Web UI, like: localhost:4040
> 4. spark-submit will never finish, you will have to kill the process
> Cause: The application creates a thread with infinite loop for web UI communication 
and never stops it. The application is waiting for the thread to be finished instead, even
if you close the web page.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message