Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Sun, 27 Aug 2017 23:06:00 +0000 (UTC)
From: "Junping Du (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13097050.1503501320000.132080.1503875160108@Atlassian.JIRA>
In-Reply-To: <JIRA.13097050.1503501320000@Atlassian.JIRA>
References: <JIRA.13097050.1503501320000@Atlassian.JIRA> <JIRA.13097050.1503501320626@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-7083) Log aggregation deletes/renames
 while file is open
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sun, 27 Aug 2017 23:06:05 -0000


    [ https://issues.apache.org/jira/browse/YARN-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143247#comment-16143247 ] 

Junping Du commented on YARN-7083:
----------------------------------

Thanks [~daryn] for reporting the issue and [~jlowe] for delivering a fix. The issue reported here is we were losing writer.close() in proper places since YARN-6288 as we were fully lean on auto close in the end. The fix here seems straightforward enough so I think no UT here should be fine. 
+1. I will commit it tomorrow if no further comments.

> Log aggregation deletes/renames while file is open
> --------------------------------------------------
>
>                 Key: YARN-7083
>                 URL: https://issues.apache.org/jira/browse/YARN-7083
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.8.2
>            Reporter: Daryn Sharp
>            Assignee: Jason Lowe
>            Priority: Critical
>         Attachments: YARN-7083.001.patch
>
>
> YARN-6288 changes the log aggregation writer to be an autoclosable.  Unfortunately the try-with-resources block for the writer will either rename or delete the log while open.
> Assuming the NM's behavior is correct, deleting open files only results in ominous WARNs in the nodemanager log and increases the rate of logging in the NN when the implicit try-with-resource close fails.  These red herrings complicate debugging efforts.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org