spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "duyanghao (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-21348) Spark history Server load too slow when eventlog is large
Date Sat, 08 Jul 2017 02:22:00 GMT
duyanghao created SPARK-21348:
---------------------------------

             Summary: Spark history Server load too slow when eventlog is large
                 Key: SPARK-21348
                 URL: https://issues.apache.org/jira/browse/SPARK-21348
             Project: Spark
          Issue Type: Bug
          Components: Web UI
    Affects Versions: 2.1.0
            Reporter: duyanghao
            Priority: Critical


There is a eventlog file in hdfs as below:
```
-rwxrwx---   3 root supergroup 9634238709 2017-07-08 04:37 hdfs://xxxxx/shared/spark-logs/xxxxx/xxxx.lz4
```

and i start a spark history server to loading that eventlog file,but it is too slow to get
spark web ui through that spark history server.

it has requested for several hours but nothing about spark application web ui appears in the
browser

And the usage of spark history server is 105.3%

The host is 48 Cores+130G Memory,enough Computing resources for spark history server.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message