hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2928) Application Timeline Server (ATS) next gen: phase 1
Date Wed, 25 Feb 2015 23:09:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337419#comment-14337419

Gopal V commented on YARN-2928:

The original discussion about ATS v1 drew inspiration from existing systems like rsyslog and
scribe, which are simple systems which buffer/route/forward into a central store.

Those mechanisms were very useful in duplicating higher priority (and rare) events for immediate
alerting/dashboards (errors/sec etc).

Are there any plans to include intermediate routing/forwarding systems for ATS v2?

The "tail -f | grep" firehose across a cluster is useful in avoiding scalability issues when
looking for rare events in a distributed store.

Being able to route something like a node blacklisting event from an AppMaster to such a system
would prevent the fault checker systems from having to produce irrelevant ATS traffic periodically
to scrape through it.

> Application Timeline Server (ATS) next gen: phase 1
> ---------------------------------------------------
>                 Key: YARN-2928
>                 URL: https://issues.apache.org/jira/browse/YARN-2928
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>            Priority: Critical
>         Attachments: ATSv2.rev1.pdf, ATSv2.rev2.pdf, Data model proposal v1.pdf
> We have the application timeline server implemented in yarn per YARN-1530 and YARN-321.
Although it is a great feature, we have recognized several critical issues and features that
need to be addressed.
> This JIRA proposes the design and implementation changes to address those. This is phase
1 of this effort.

This message was sent by Atlassian JIRA

View raw message