hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7083) Log aggregation deletes/renames while file is open
Date Sun, 27 Aug 2017 23:06:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143247#comment-16143247
] 

Junping Du commented on YARN-7083:
----------------------------------

Thanks [~daryn] for reporting the issue and [~jlowe] for delivering a fix. The issue reported
here is we were losing writer.close() in proper places since YARN-6288 as we were fully lean
on auto close in the end. The fix here seems straightforward enough so I think no UT here
should be fine. 
+1. I will commit it tomorrow if no further comments.

> Log aggregation deletes/renames while file is open
> --------------------------------------------------
>
>                 Key: YARN-7083
>                 URL: https://issues.apache.org/jira/browse/YARN-7083
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.8.2
>            Reporter: Daryn Sharp
>            Assignee: Jason Lowe
>            Priority: Critical
>         Attachments: YARN-7083.001.patch
>
>
> YARN-6288 changes the log aggregation writer to be an autoclosable.  Unfortunately the
try-with-resources block for the writer will either rename or delete the log while open.
> Assuming the NM's behavior is correct, deleting open files only results in ominous WARNs
in the nodemanager log and increases the rate of logging in the NN when the implicit try-with-resource
close fails.  These red herrings complicate debugging efforts.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message