hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node
Date Thu, 02 Jun 2016 01:10:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311530#comment-15311530
] 

Arun Suresh edited comment on YARN-5171 at 6/2/16 1:10 AM:
-----------------------------------------------------------

Thanks for the patch [~elgoiri]..

The general approach looks good, We would also need to ensure the following though :
# Since the container is created outside the RM, the {{attemptResourceUsage}} field in the
{{SchedulerApplicationAttempt}} is not incremented (This is done by the specific schedulers
and is found in the specific implementation of the SchedulerApplicationAttempt). I guess we
should have a separate {{attemptResourceUsageOpportunistic}} field introduced in the SchedulerApplicationAttempt
class, that is incremented when the DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the {{AbstractYarnScheduler}}
to also decrement the {{attemptResourceUsageOpportunistic}} of the corresponding app attempt
if the completed container was opportunistic (we can get that from the {{ContainerStatus}}).
We don't have to worry about NOT decrementing {{attemptResourceUsage}}, since that is done
in the specific Scheduler, and we are modifying the {{AbstractYarnScheduler::completedContainer()}}.


was (Author: asuresh):
Thanks for the patch [~elgoiri]..

The general approach looks good, We would also need to ensure the following though :
# Since the container is created outside the RM, the {{attemptResourceUsage}} field in the
{{SchedulerApplicationAttempt}} is not incremented (This is done by the specific schedulers
and is found in the specific implementation of the SchedulerApplicationAttempt). I guess we
should have a separate {{attemptResourceUsageOpportunistic}} field introduced in the SchedulerApplicationAttempt
class, that is incremented when the DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the {{AbstractYarnScheduler}}
to also decrement the {{attemptResourceUsageOpportunistic}} of the corresponding app attempt
if the completed container was opportunistic (we can get that from the {{ContainerStatus}})

> Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-5171
>                 URL: https://issues.apache.org/jira/browse/YARN-5171
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Inigo Goiri
>         Attachments: YARN-5171.000.patch, YARN-5171.001.patch
>
>
> Currently, the RM does not know about Containers allocated by the OpportunisticContainerAllocator
on the NM. This JIRA proposes to extend the Distributed Scheduler request interceptor and
the protocol to notify the RM of new containers as and when they are allocated at the NM.
The {{RMContainer}} should also be extended to expose the {{ExecutionType}} of the container.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message