hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate "long lived"
Date Tue, 24 Jun 2014 17:03:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042370#comment-14042370
] 

Steve Loughran commented on YARN-1039:
--------------------------------------

bq. We need to have another flag to indicate that the containers requested by an AM will be
long lived, it must be set at application creation time and all containers of the app will
be considered long lived. This is because the RM does not keep track of individual container
requests.

I see this, but disagree as it doesn't meet all use cases. For different requests we may want:
long-lived, pre-emptible, anti-affine, .... 

This can't go in requests -as you point out- but we already have a per-request flag that really
sets a bit in the priority level -the lax placement option. 

If the other requests set the values at that priority then it is similar.

Even so, setting these values in a request is confusing -even today. It would be better to
have some operation to get/set the attributes of a priority for requests. 

This would be a bigger change...something we my not want to rush into. 

> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>
>                 Key: YARN-1039
>                 URL: https://issues.apache.org/jira/browse/YARN-1039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Steve Loughran
>            Assignee: Craig Welch
>         Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
>
>
> A container request could support a new parameter "long-lived". This could be used by
a scheduler that would know not to host the service on a transient (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived containers
on the same node



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message