hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4183) Enabling generic application history forces every job to get a timeline service delegation token
Date Thu, 19 Nov 2015 23:07:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014685#comment-15014685

Sangjin Lee commented on YARN-4183:

Thanks [~vinodkv] for the suggestion. The following is my proposal.

- yarn.timeline-service.enabled should be interpreted as the config that indicates the timeline
service daemon is up *both* on the client side and the server side
- specifically for the RM's system metrics publisher, it should continue to check both yarn.timeline-service.enabled
and yarn.resourcemanager.system-metrics-publisher.enabled (which is the current code); if
not, and the timeline service is not up (which is a totally valid situation), the system metrics
publisher will have a continuous stream of write errors
- there needs to be a strong client-side config for each framework (MR or tez) that controls
whether it wants to use the timeline service; the client can use the timeline service only
if *both* yarn.timeline-service.enabled is true and its own config is true

I hope the last point should address the original issue of this JIRA (if MR does not want
to use the timeline service, it should be able to do so).

The decision to use the timeline service and get the delegation token should not hinge on
which version is enabled IMO, as the version is another global property.

Let me know if that sounds reasonable.

> Enabling generic application history forces every job to get a timeline service delegation
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-4183
>                 URL: https://issues.apache.org/jira/browse/YARN-4183
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Mit Desai
>            Assignee: Mit Desai
>         Attachments: YARN-4183.1.patch
> When enabling just the Generic History Server and not the timeline server, the system
metrics publisher will not publish the events to the timeline store as it checks if the timeline
server and system metrics publisher are enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force every yarn
application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if application
history server is enabled.

This message was sent by Atlassian JIRA

View raw message