hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3476) Nodemanager can fail to delete local logs if log aggregation fails
Date Fri, 10 Apr 2015 19:42:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490233#comment-14490233
] 

Jason Lowe commented on YARN-3476:
----------------------------------

bq. Such cases we can use the aggregated log-retention configuration i.e 'yarn.log-aggregation.retain-seconds'
for deleting from disk.

No, that's not going to be OK.  One is the lifetime on the aggregation filesystem and what
we're considering is the lifetime on the local disk.  I can hold a lot more logs in HDFS than
I can in the local disk.

> Nodemanager can fail to delete local logs if log aggregation fails
> ------------------------------------------------------------------
>
>                 Key: YARN-3476
>                 URL: https://issues.apache.org/jira/browse/YARN-3476
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: log-aggregation, nodemanager
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Rohith
>
> If log aggregation encounters an error trying to upload the file then the underlying
TFile can throw an illegalstateexception which will bubble up through the top of the thread
and prevent the application logs from being deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message