hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.
Date Thu, 25 Jun 2015 06:08:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600754#comment-14600754
] 

Xuan Gong commented on YARN-221:
--------------------------------

bq. we want to be able to config the sample rate without code change. If it isn't in yarn-site.xml,
where should we store the value?

If the default policy is SampleRateContainerLogAggregationPolicy, we already have the default
value. If the users want to change the value (sample rate), they could set through ASC#logaggregationContext#setParameter().

If we set the parameter in yarn-site.xml, all the applications will be affected. Since this
is per application, so I think that this probably will be suitable. Thoughts ?

> NM should provide a way for AM to tell it not to aggregate logs.
> ----------------------------------------------------------------
>
>                 Key: YARN-221
>                 URL: https://issues.apache.org/jira/browse/YARN-221
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation, nodemanager
>            Reporter: Robert Joseph Evans
>            Assignee: Ming Ma
>         Attachments: YARN-221-trunk-v1.patch, YARN-221-trunk-v2.patch, YARN-221-trunk-v3.patch,
YARN-221-trunk-v4.patch, YARN-221-trunk-v5.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the logs should
not be aggregated, that they should be aggregated with a high priority, or that they should
be aggregated but with a lower priority.  The AM should be able to do this in the ContainerLaunch
context to provide a default value, but should also be able to update the value when the container
is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid connection
to the NN at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message