hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM
Date Thu, 09 Jul 2015 22:56:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621416#comment-14621416

Konstantinos Karanasos commented on YARN-3116:

Thanks for the feedback, [~giovanni.fumarola] and [~zjshen].
I agree that the two JIRAs are complementary.

One first observation is that probably we should have two different flavours of container
types: ContainerRequestType and ContainerType. The former should be exposed to the application
(e.g., centralized or distributed scheduling), while the latter should encode information
for the NM to properly handle the container request without exposing it to the application
(e.g., GUARANTEED or QUEUEABLE, AM_CONTAINER or "non-AM"_CONTAINER). This looks like a more
future-proof design.

That said, I am wondering whether the ContainerTokenIdentifier is the proper place to set
the container type to accommodate other types of containers (and not just the isAMContainer
flag). I am on the go today without access to the code -- will get back to you tomorrow regarding

> [Collector wireup] We need an assured way to determine if a container is an AM container
on NM
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-3116
>                 URL: https://issues.apache.org/jira/browse/YARN-3116
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Giovanni Matteo Fumarola
>         Attachments: YARN-3116.patch, YARN-3116.v2.patch, YARN-3116.v3.patch, YARN-3116.v4.patch,
YARN-3116.v5.patch, YARN-3116.v6.patch, YARN-3116.v7.patch, YARN-3116.v8.patch
> In YARN-3030, to start the per-app aggregator only for a started AM container,  we need
to determine if the container is an AM container or not from the context in NM (we can do
it on RM). This information is missing, such that we worked around to considered the container
with ID "xxxx_01" as the AM container. Unfortunately, this is neither necessary or sufficient
condition. We need to have a way to determine if a container is an AM container on NM. We
can add flag to the container object or create an API to do the judgement. Perhaps the distributed
AM information may also be useful to YARN-2877.

This message was sent by Atlassian JIRA

View raw message