hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
Date Fri, 17 Jun 2016 22:16:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337102#comment-15337102

Sangjin Lee commented on YARN-5174:

I am fine with the current statement. My point earlier was whether we should draw more attention
to this than is needed.

Individual frameworks can do either of the approaches you pointed out. I suspect most frameworks
already aggregate values at the app-equivalent level and are capable of emitting already-aggregated
metrics at the YARN application entity. I hope the other option is more of an exception.

Say, in case of Spark on YARN, multiple spark executors will be launched (with each one executing
multiple tasks). So, if they ever integrate with ATSv2 and report say some executor level
metrics, then we can make it clear that they themselves will have to roll over these metrics
to app level. This will not be an immediate use case though, if ever.
We'd like Spark to use timeline service v.2 as soon as feasible. :) Seriously, I think it
would be easier for them to emit aggregated metrics themselves? Otherwise, they would need
to identify a suitable entity of their own and use this aggregation ops. Either should be

> several updates/corrections to timeline service documentation
> -------------------------------------------------------------
>                 Key: YARN-5174
>                 URL: https://issues.apache.org/jira/browse/YARN-5174
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>              Labels: yarn-2928-1st-milestone
>         Attachments: Hierarchy.png, PublishingApplicationDatatoYARNTimelineServicev.pdf,
The YARN Timeline Service v.2.pdf, YARN-5174-YARN-2928.01.patch, YARN-5174-YARN-2928.02.patch,
YARN-5174-YARN-2928.03.patch, YARN-5174-YARN-2928.03.patch, YARN-5174-YARN-2928.04.patch,
YARN-5174-YARN-2928.05.patch, flow_hierarchy.png
> One part that is missing in the documentation is the need to add {{hbase-site.xml}} on
the client side (the client hadoop cluster). First, we need to arrive at the minimally required
client setting to connect to the right hbase cluster. Then, we need to document it so that
users know exactly what to do to configure the cluster to use the timeline service v.2.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message