hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5302) Yarn Application log Aggreagation fails due to NM can not get correct HDFS delegation token II
Date Fri, 01 Jul 2016 07:31:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358553#comment-15358553
] 

Varun Saxena edited comment on YARN-5302 at 7/1/16 7:30 AM:
------------------------------------------------------------

Ok. So this happens when RM has renewed the token but not yet passed the token onto the NM
in Heartbeat because NM restarted.
Do we not update the token in NM state store when it changes ?


was (Author: varun_saxena):
Ok. So this happens when RM has renewed the token but not yet passed the token onto the NM
in Heartbeat because NM restarted.

> Yarn Application log Aggreagation fails due to NM can not get correct HDFS delegation
token II
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-5302
>                 URL: https://issues.apache.org/jira/browse/YARN-5302
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>            Reporter: Xianyin Xin
>
> Different with YARN-5098, this happens at NM side. When NM recovers, credentials are
read from NMStateStore. When initialize app aggregators, exception happens because of the
overdue tokens. The app is a long running service.
> {code:title=LogAggregationService.java}
>   protected void initAppAggregator(final ApplicationId appId, String user,
>       Credentials credentials, ContainerLogsRetentionPolicy logRetentionPolicy,
>       Map<ApplicationAccessType, String> appAcls,
>       LogAggregationContext logAggregationContext) {
>     // Get user's FileSystem credentials
>     final UserGroupInformation userUgi =
>         UserGroupInformation.createRemoteUser(user);
>     if (credentials != null) {
>       userUgi.addCredentials(credentials);
>     }
>    ...
>     try {
>       // Create the app dir
>       createAppDir(user, appId, userUgi);
>     } catch (Exception e) {
>       appLogAggregator.disableLogAggregation();
>       if (!(e instanceof YarnRuntimeException)) {
>         appDirException = new YarnRuntimeException(e);
>       } else {
>         appDirException = (YarnRuntimeException)e;
>       }
>       appLogAggregators.remove(appId);
>       closeFileSystems(userUgi);
>       throw appDirException;
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message