tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-2073) SimpleHistoryLoggingService cannot be read by log aggregation (umask)
Date Tue, 10 Feb 2015 23:36:11 GMT

    [ https://issues.apache.org/jira/browse/TEZ-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315185#comment-14315185

Vinod Kumar Vavilapalli commented on TEZ-2073:

bq. Is this something that also needs a fix in YARN? Or are there standard guidelines on what
an app can/cannot do or should behave in a certain way?
YARN cannot aggregate logs if logs generated by user apps don't have the 'right' permissions
for YARN to be able to read them.

bq. It may be ideal for YARN to provide the proper log directory umask settings so that YARN
applications can write file that can be aggregated correctly.
Umask is tied to processes not directories. And since the processes are user code, YARN cannot
control it.

> SimpleHistoryLoggingService cannot be read by log aggregation (umask)
> ---------------------------------------------------------------------
>                 Key: TEZ-2073
>                 URL: https://issues.apache.org/jira/browse/TEZ-2073
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jonathan Eagles
> Nodemanager logs have the following log aggregation error.
> In a system where default fs.permissions.umask-mode=077. All file can be aggregated except
the Simple History file since it writes files using the system's default FileSystem umask.
> {quote}
> [LogAggregationService #278] ERROR logaggregation.AggregatedLogFormat: Error aggregating
log file. Log file : /tmp/yarn-logs/application_1423009000687_15308/container_e04_1423009000687_15308_01_000001/history.txt.appattempt_1423009000687_15308_000001.
(Permission denied)

This message was sent by Atlassian JIRA

View raw message