spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bozarth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history
Date Thu, 25 Aug 2016 21:12:20 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437682#comment-15437682
] 

Alex Bozarth commented on SPARK-17243:
--------------------------------------

[~wgtmac] until this is fixed you can limit the number of applications available by setting
`spark.history.retainedApplications` It limits the apps the history server loads

> Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+
application history
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-17243
>                 URL: https://issues.apache.org/jira/browse/SPARK-17243
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 2.0.0
>         Environment: Linux
>            Reporter: Gang Wu
>
> The summary page of Spark 2.0 history server web UI keep displaying "Loading history
summary..." all the time and crashes the browser when there are more than 10K application
history event logs on HDFS. 
> I did some investigation, "historypage.js" file sends a REST request to /api/v1/applications
endpoint of history server REST endpoint and gets back json response. When there are more
than 10K applications inside the event log directory it takes forever to parse them and render
the page. When there are only hundreds or thousands of application history it is running fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message