spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Li (Jira)" <>
Subject [jira] [Updated] (SPARK-18486) Access of SparkUI after task completion
Date Sun, 17 May 2020 17:58:00 GMT


Xiao Li updated SPARK-18486:
    Component/s:     (was: Optimizer)

> Access of SparkUI after task completion
> ---------------------------------------
>                 Key: SPARK-18486
>                 URL:
>             Project: Spark
>          Issue Type: Question
>          Components: Structured Streaming, Web UI
>    Affects Versions: 2.0.1
>         Environment: Windows server 2008 R2
> Spark Version: 2.0.1
> Running locally and accessing sparkUI on por 4040
>            Reporter: Bhanuday Birla
>            Priority: Minor
>              Labels: newbie, performance
>   Original Estimate: 168h
>  Remaining Estimate: 168h
> Hi, 
> I am running spark scripts in on local machine. In order to see completion times of different
jobs, I want to access SparkUI on port 4040. But i can access it while the script is running.
After completion of the script, it's not accessible. To resolve this I have set spark.eventLog.enabled
property to true inside my application. 
> conf =SparkConf().setMaster("local[4]").setAppName("IAP").set("spark.eventLog.enabled",
> Also I have created a folder named "spark-events" storing history logs which can be picked
up by SparkUI. But still no luck. I am working on a project in my firm and get stuck here.
Please help me in any possible way.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message