hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4766) NM should not aggregate logs older than the retention policy
Date Thu, 19 May 2016 21:50:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292217#comment-15292217
] 

Haibo Chen commented on YARN-4766:
----------------------------------

Thanks Ray for your comment, the name you suggested is much better. Made the change and upload
a new patch to verify the patch still works since it is been sitting for a while.

> NM should not aggregate logs older than the retention policy
> ------------------------------------------------------------
>
>                 Key: YARN-4766
>                 URL: https://issues.apache.org/jira/browse/YARN-4766
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: log-aggregation, nodemanager
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>         Attachments: yarn4766.001.patch, yarn4766.002.patch, yarn4766.003.patch, yarn4766.004.patch,
yarn4766.004.patch
>
>
> When a log aggregation fails on the NM the information is for the attempt is kept in
the recovery DB. Log aggregation can fail for multiple reasons which are often related to
HDFS space or permissions.
> On restart the recovery DB is read and if an application attempt needs its logs aggregated,
the files are scheduled for aggregation without any checks. The log files could be older than
the retention limit in which case we should not aggregate them but immediately mark them for
deletion from the local file system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message