hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4233) YARN Timeline Service plugin: ATS v1.5
Date Thu, 15 Oct 2015 16:22:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959160#comment-14959160

Jason Lowe commented on YARN-4233:

Most of the details can be found in the discussion in YARN-3942.  Executive summary is that
the original ATS has scaling issues, and we need an interim solution until the [ATSv2 effort|https://issues.apache.org/jira/browse/YARN-2928]
is ready.  The basic idea is to have clients and jobs write entities to HDFS rather than posting
them to a single, central web service to better handle the write load from a large cluster
running lots of jobs concurrently.  We need something like this to get a working Tez UI on
our large clusters and can't wait until ATSv2 is ready.

> YARN Timeline Service plugin: ATS v1.5
> --------------------------------------
>                 Key: YARN-4233
>                 URL: https://issues.apache.org/jira/browse/YARN-4233
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
> Copy the description from YARN-3942: Timeline store to read events from HDFS
> This adds a new timeline store plugin that is intended as a stop-gap measure to mitigate
some of the issues we've seen with ATS v1 while waiting for ATS v2. The intent of this plugin
is to provide a workable solution for running the Tez UI against the timeline server on a
large-scale clusters running many thousands of jobs per day.

This message was sent by Atlassian JIRA

View raw message