Mailing-List: contact issues-help@tez.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@tez.apache.org
Date: Tue, 10 Feb 2015 23:36:11 +0000 (UTC)
From: "Vinod Kumar Vavilapalli (JIRA)" <jira@apache.org>
To: issues@tez.apache.org
Message-ID: <JIRA.12773990.1423606048000.6978.1423611371742@Atlassian.JIRA>
In-Reply-To: <JIRA.12773990.1423606048000@Atlassian.JIRA>
References: <JIRA.12773990.1423606048000@Atlassian.JIRA>
 <JIRA.12773990.1423606048847@arcas>
Subject: [jira] [Commented] (TEZ-2073) SimpleHistoryLoggingService cannot be
 read by log aggregation (umask)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/TEZ-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315185#comment-14315185 ] 

Vinod Kumar Vavilapalli commented on TEZ-2073:
----------------------------------------------

bq. Is this something that also needs a fix in YARN? Or are there standard guidelines on what an app can/cannot do or should behave in a certain way?
YARN cannot aggregate logs if logs generated by user apps don't have the 'right' permissions for YARN to be able to read them.

bq. It may be ideal for YARN to provide the proper log directory umask settings so that YARN applications can write file that can be aggregated correctly.
Umask is tied to processes not directories. And since the processes are user code, YARN cannot control it.

> SimpleHistoryLoggingService cannot be read by log aggregation (umask)
> ---------------------------------------------------------------------
>
>                 Key: TEZ-2073
>                 URL: https://issues.apache.org/jira/browse/TEZ-2073
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jonathan Eagles
>
> Nodemanager logs have the following log aggregation error.
> In a system where default fs.permissions.umask-mode=077. All file can be aggregated except the Simple History file since it writes files using the system's default FileSystem umask.
> {quote}
> [LogAggregationService #278] ERROR logaggregation.AggregatedLogFormat: Error aggregating log file. Log file : /tmp/yarn-logs/application_1423009000687_15308/container_e04_1423009000687_15308_01_000001/history.txt.appattempt_1423009000687_15308_000001. /tmp/yarn-logs/application_1423009000687_15308/container_e04_1423009000687_15308_01_000001/history.txt.appattempt_1423009000687_15308_000001 (Permission denied)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)