hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC
Date Wed, 23 Dec 2015 06:29:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069226#comment-15069226

Arun Suresh commented on YARN-2882:

bq. However, I think we still need a flag in ResourceRequest to describe if it's a "opportunistic"
container or not, correct? Otherwise RM/LocalRM cannot decide if it can take risk to allocate
a queueable/oversubscribe container.
[~leftnoteasy], as [~kasha@cloudera.com] had mentioned, we had a pretty lengthy discussion
about this.. finally we came to the consensus that we dissallow the AM from making that decision.
# Firstly, The AM might not be in a position to know if a Request can be handled in a guaranteed
or opportunistic manner. Having a pluggable policy on the NM that can filter some Resource
Requests (for eg. Distributed Scheduling can first target non-strict locality requests) which
can evolve to really dynamic policies based on load etc.
# Secondly, this would ensure that delinquent AMs wont game the scheduler by asking for only
guaranteed Resources. 
Hope this made sense....

> Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC
> -----------------------------------------------------------------------------------
>                 Key: YARN-2882
>                 URL: https://issues.apache.org/jira/browse/YARN-2882
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Konstantinos Karanasos
>         Attachments: YARN-2882-yarn-2877.001.patch, YARN-2882-yarn-2877.002.patch, YARN-2882-yarn-2877.003.patch,
YARN-2882-yarn-2877.004.patch, yarn-2882.patch
> This JIRA introduces the notion of container types.
> We propose two initial types of containers: guaranteed-start and queueable containers.
> Guaranteed-start are the existing containers, which are allocated by the central RM and
are instantaneously started, once allocated.
> Queueable is a new type of container, which allows containers to be queued in the NM,
thus their execution may be arbitrarily delayed.

This message was sent by Atlassian JIRA

View raw message