hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7124) LogAggregationTFileController deletes/renames while file is open
Date Tue, 10 Oct 2017 22:51:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe updated YARN-7124:
    Attachment: YARN-7124.001.patch

This isn't pretty, but it's small and I think it will do the trick.  It does imply that calling
postWrite means it is not OK to call write afterwards, and only close() can be called once
postWrite is called.  That's what the code is doing now anyway, but this would codify that.

[~djp] [~xgong] please take a look.

> LogAggregationTFileController deletes/renames while file is open
> ----------------------------------------------------------------
>                 Key: YARN-7124
>                 URL: https://issues.apache.org/jira/browse/YARN-7124
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0, 3.0.0-beta1
>            Reporter: Daryn Sharp
>            Assignee: Jason Lowe
>            Priority: Critical
>         Attachments: YARN-7124.001.patch
> YARN-6288 changes the log aggregation writer to be an autoclosable.  Unfortunately the
try-with-resources block for the writer will either rename or delete the log while open.
> Assuming the NM's behavior is correct, deleting open files only results in ominous WARNs
in the nodemanager log and increases the rate of logging in the NN when the implicit try-with-resource
close fails.  These red herrings complicate debugging efforts.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message