hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7087) NM failed to perform log aggregation due to absent container
Date Thu, 24 Aug 2017 16:20:01 GMT

     [ https://issues.apache.org/jira/browse/YARN-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe updated YARN-7087:
    Attachment: YARN-7087.001.patch

Attaching a patch that adds the container type to the log aggregation container finished event
which eliminates the need for AppLogAggregatorImpl to lookup the container in the context
and potentially not find it.

This appears to be occurring quite often on our clusters in cases where an application is
killed, so it would be great to fix this for 2.8.2.

> NM failed to perform log aggregation due to absent container
> ------------------------------------------------------------
>                 Key: YARN-7087
>                 URL: https://issues.apache.org/jira/browse/YARN-7087
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: log-aggregation
>    Affects Versions: 2.8.1
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Critical
>         Attachments: YARN-7087.001.patch
> Saw a case where the NM failed to aggregate the logs for a container because it claimed
it was absent:
> {noformat}
> 2017-08-23 18:35:38,283 [AsyncDispatcher event handler] WARN logaggregation.LogAggregationService:
Log aggregation cannot be started for container_e07_1503326514161_502342_01_000001, as its
an absent container
> {noformat}
> Containers should not be allowed to disappear if they're not done being fully processed
by the NM.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message