hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.
Date Fri, 07 Aug 2015 00:57:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661140#comment-14661140
] 

Ming Ma commented on YARN-221:
------------------------------

My main motivation of reusing ContainerTerminationContext is to simplify YARN API if possible.
The context around container could be abstracted into a common interface for both AuxiliaryService
and ContainerLogAggregationPolicy. It is possible some YARN application might want to create
its own ContainerLogAggregationPolicy; in which case they just need to know about ContainerTerminationContext.

Sometimes there is no right or wrong answer when it comes to API design. In this case, ContainerTokenIdentifier
or ContainerTerminationContext or to-be-defined ContainerLogContext will work for the current
scenarios. Given this isn't a public interface, we can use ContainerTokenIdentifier until
new scenarios come up. Thoughts?

> NM should provide a way for AM to tell it not to aggregate logs.
> ----------------------------------------------------------------
>
>                 Key: YARN-221
>                 URL: https://issues.apache.org/jira/browse/YARN-221
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation, nodemanager
>            Reporter: Robert Joseph Evans
>            Assignee: Ming Ma
>         Attachments: YARN-221-6.patch, YARN-221-7.patch, YARN-221-8.patch, YARN-221-trunk-v1.patch,
YARN-221-trunk-v2.patch, YARN-221-trunk-v3.patch, YARN-221-trunk-v4.patch, YARN-221-trunk-v5.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the logs should
not be aggregated, that they should be aggregated with a high priority, or that they should
be aggregated but with a lower priority.  The AM should be able to do this in the ContainerLaunch
context to provide a default value, but should also be able to update the value when the container
is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid connection
to the NN at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message