spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davis Shepherd (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.
Date Fri, 04 Apr 2014 00:42:16 GMT

     [ https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Davis Shepherd updated SPARK-1411:
----------------------------------

    Priority: Major  (was: Minor)

> When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.
> -------------------------------------------------------------------------------------------
>
>                 Key: SPARK-1411
>                 URL: https://issues.apache.org/jira/browse/SPARK-1411
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 0.9.0
>         Environment: Ubuntu 12.04 LTS Precise
> 4 Nodes, 
> 16 cores each, 
> 196GB RAM each.
>            Reporter: Davis Shepherd
>         Attachments: Screen Shot 2014-04-03 at 5.35.00 PM.png
>
>
> For any long running job with many stages, the web ui only shows the first n stages of
the job (where spark.ui.retainedStages=n). The most recent stages are immediately dropped
and are only visible for a brief time.  This renders the UI pretty useless after a pretty
short amount of time for a long running non-streaming job. I am unsure as to whether similar
results appear for streaming jobs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message