hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4953) Delete completed container log folder when rolling log aggregation is enabled
Date Thu, 02 Jun 2016 13:45:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-4953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312317#comment-15312317

Jason Lowe commented on YARN-4953:

Sorry for missing this earlier.  As I mentioned on YARN-5193, log aggregation originally aggregated
logs for containers as they finished.  The main issue with aggregating as containers complete
is the additional load on the namenode.  See YARN-219.  Our large clusters were getting swamped
with lease renewal load until that was changed.  We might be able to work around it with append
operations, but it can be very problematic to simply have the NM hold the aggregated log file
open until the app completes.

> Delete completed container log folder when rolling log aggregation is enabled
> -----------------------------------------------------------------------------
>                 Key: YARN-4953
>                 URL: https://issues.apache.org/jira/browse/YARN-4953
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
> There would be potential bottle neck when cluster is running with very large number of
containers on the same NodeManager for single application. The linux limits the subfolders
count to 32K. If number of containers is greater than 32K for an application, there would
be container launch failure. At this point of time, there are no more containers can be launched
in this node.
> Currently log folders are deleted after app is finished. Rolling log aggregation aggregates
logs to hdfs periodically. 
> I think if aggregation is completed for finished containers, then clean up can be done
i.e deleting log folder for finished containers. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message