hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4183) Enabling generic application history forces every job to get a timeline service delegation token
Date Wed, 11 Nov 2015 00:47:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999697#comment-14999697

Li Lu commented on YARN-4183:

Thanks [~vinodkv]! I agree we should find some better ways to organize \*.enabled in ATS (we've
got two different versions in our code base and will add two more). For end users, we need
to provide mechanisms to distinguish at least 3 versions of ATS API calls, v1, 1.5, and v2
in future. 

bq. There should be an explicit yarn.timeline-service.version which tells YarnClient to get
tokens or not - yes for non-present version (default), v1, v2 but no for v1.5.
bq.  The version field has semantics on both client and server side at the same time - it's
picking a solution end-to-end.
IIUC, the newly proposed {{yarn.timeline-service.version}} supports a sanity check mechanism:
each API should check if the current running ATS's version is equal to or higher than it's
required version. For example, when a ATS v1.5 API is called, but {{yarn.timeline-service.version}}
is set to v1, it should simply throw an exception. We can also decide if we need to get tokens
or not in YARN client by checking this version number. 

We can distinguish API versions through their names. We need to keep the V1 APIs unchanged,
but add V15 and V2 after the new APIs to clarify their API version. Inside each V15 and V2
API we can perform the sanity check. 

Let's make the {{yarn.timeline-service.version}} change here. We can modify V1.5 APIs in YARN-4233/YARN-4234
and V2 APIs as a subtask of YARN-2928. I can open a new subtask in YARN-2928 to fix this for

> Enabling generic application history forces every job to get a timeline service delegation
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-4183
>                 URL: https://issues.apache.org/jira/browse/YARN-4183
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Mit Desai
>            Assignee: Mit Desai
>             Fix For: 3.0.0, 2.8.0, 2.7.2
>         Attachments: YARN-4183.1.patch
> When enabling just the Generic History Server and not the timeline server, the system
metrics publisher will not publish the events to the timeline store as it checks if the timeline
server and system metrics publisher are enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force every yarn
application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if application
history server is enabled.

This message was sent by Atlassian JIRA

View raw message