hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
Date Tue, 08 Dec 2015 17:11:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047084#comment-15047084

Junping Du commented on YARN-3623:

I don't quite familiar with requirement of ATS v1.5. However, in stands of ATS v2, I would
agree with [~sjlee0]'s comments above to make this effect works on writer side only (TimelineClient).

More clarifications:
1. This version configuration is to benefit application/framework to have flexibility to run
on top of YARN cluster with ATS v1 or v2 running with indicating the latest stable version
ATS service that the cluster can support.  ATS v1 and v2 client are different binary bits
and use different incompatible APIs to put information like event, metrics, etc. so far. With
getting proper configuration from YARN, the application can aware the ATS service version
when landing on YARN cluster and can choose different TimelineClient to push info and get
rid of our pains in doing TestDistributedCache for v1/v2 timeline service.

2. We shouldn't break rolling upgrade scenario, or it could be seen as incompatible feature
which cannot land on 2.x branch. That also means, we should support ATS v1 and v2 services
at the same time during cluster upgrade so legacy/existing applications can still access their
old ATS service which is the same as many rollup stories. 

2 clarification is more related to this change: we'd better change "yarn.timeline-service.version"
to "yarn.timeline-service.latest.version" and use "indicate to clients what is the latest
stable version of the running timeline service" to get rid of any confusion here. Also it
is better to explicitly mention that our support range for ATS is: [X-1, X] for rolling upgrade
(assume X is latest stable ATS version).

> We should have a config to indicate the Timeline Service version
> ----------------------------------------------------------------
>                 Key: YARN-3623
>                 URL: https://issues.apache.org/jira/browse/YARN-3623
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Xuan Gong
>         Attachments: YARN-3623-2015-11-19.1.patch
> So far RM, MR AM, DA AM added/changed new config to enable the feature to write the timeline
data to v2 server. It's good to have a YARN timeline-service.version config like timeline-service.enable
to indicate the version of the running timeline service with the given YARN cluster. It's
beneficial for users to more smoothly move from v1 to v2, as they don't need to change the
existing config, but switch this config from v1 to v2. And each framework doesn't need to
have their own v1/v2 config.

This message was sent by Atlassian JIRA

View raw message