hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM
Date Fri, 19 Jun 2015 20:51:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14593944#comment-14593944

Zhijie Shen commented on YARN-3116:

[~giovanni.fumarola], thanks for the patch. Two quick comments about it:

1. IMHO, I prefer to use ContainerTokenIdentifier instead of ContainerLaunchContext to pass
the AM flag, as we did for node label and aggregation log context. ContainerLaunchContext
is user API, such that user has the freedom to change the flag and says whether the container
he starts is AM or not. Theoretically, user can still change the container token too, but
it's not that obvious to user. Thoughts?

2. Not sure if AMRMProxy wants the AM flag to be propagated somewhere, but in our use case,
once NM receives the flag, we want to further push it to ContainerInitializationContext, such
that we can get the flag inside aux service.

> [Collector wireup] We need an assured way to determine if a container is an AM container
on NM
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-3116
>                 URL: https://issues.apache.org/jira/browse/YARN-3116
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Giovanni Matteo Fumarola
>         Attachments: YARN-3116.patch
> In YARN-3030, to start the per-app aggregator only for a started AM container,  we need
to determine if the container is an AM container or not from the context in NM (we can do
it on RM). This information is missing, such that we worked around to considered the container
with ID "xxxx_01" as the AM container. Unfortunately, this is neither necessary or sufficient
condition. We need to have a way to determine if a container is an AM container on NM. We
can add flag to the container object or create an API to do the judgement. Perhaps the distributed
AM information may also be useful to YARN-2877.

This message was sent by Atlassian JIRA

View raw message