spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bozarth (JIRA)" <>
Subject [jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications
Date Tue, 30 Jan 2018 01:13:00 GMT


Alex Bozarth commented on SPARK-18085:

[~vanzin] since this is complete and going into 2.3 I was hoping you could help write a short
overview of the SPIP, like what's changed and why. Given how much this evolved during the
project I'm not sure if the original pitch is the best description anymore and I would like
to have a nice summary to describe the project's impact. Thanks

> SPIP: Better History Server scalability for many / large applications
> ---------------------------------------------------------------------
>                 Key: SPARK-18085
>                 URL:
>             Project: Spark
>          Issue Type: Umbrella
>          Components: Spark Core, Web UI
>    Affects Versions: 2.0.0
>            Reporter: Marcelo Vanzin
>            Priority: Major
>              Labels: SPIP
>             Fix For: 2.3.0
>         Attachments: screenshot-1.png, screenshot-2.png, spark_hs_next_gen.pdf
> It's a known fact that the History Server currently has some annoying issues when serving
lots of applications, and when serving large applications.
> I'm filing this umbrella to track work related to addressing those issues. I'll be attaching
a document shortly describing the issues and suggesting a path to how to solve them.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message