spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Masiero Vanzin (Jira)" <>
Subject [jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server
Date Tue, 21 Apr 2020 23:34:00 GMT


Marcelo Masiero Vanzin commented on SPARK-1537:

[~templedf] sorry forgot to reply.

ATSv1 wasn't a good match for this, and by the time ATSv2 was developed, interest in this
feature had long lost traction in the Spark community. So this was closed.

Also you probably can do this without requiring the code to live in Spark.

But if you actually want to contribute the integration, there's nothing preventing you from
opening a new bug and posting a PR.

> Add integration with Yarn's Application Timeline Server
> -------------------------------------------------------
>                 Key: SPARK-1537
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: YARN
>            Reporter: Marcelo Masiero Vanzin
>            Priority: Major
>         Attachments: SPARK-1537.txt, spark-1573.patch
> It would be nice to have Spark integrate with Yarn's Application Timeline Server (see
YARN-321, YARN-1530). This would allow users running Spark on Yarn to have a single place
to go for all their history needs, and avoid having to manage a separate service (Spark's
built-in server).
> At the moment, there's a working version of the ATS in the Hadoop 2.4 branch, although
there is still some ongoing work. But the basics are there, and I wouldn't expect them to
change (much) at this point.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message