hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate "long lived"
Date Wed, 21 Jan 2015 10:11:36 GMT

    [ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285449#comment-14285449
] 

Steve Loughran commented on YARN-1039:
--------------------------------------

bq. However, this can be done either based on historical information (previous waves of this
task type or previous execution of the same job) or on application level knowledge.

Historical information is generally the best estimate, though if the input data is different,
so can duration.

Maybe a simple enum as "short-lived", "session", and "service": services provide no termination,
session = "a few hours to a few days (i.e within the lifespan of kerberos tokens). 

> Add parameter for YARN resource requests to indicate "long lived"
> -----------------------------------------------------------------
>
>                 Key: YARN-1039
>                 URL: https://issues.apache.org/jira/browse/YARN-1039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Steve Loughran
>            Assignee: Craig Welch
>         Attachments: YARN-1039.1.patch, YARN-1039.2.patch, YARN-1039.3.patch
>
>
> A container request could support a new parameter "long-lived". This could be used by
a scheduler that would know not to host the service on a transient (cloud: spot priced) node.
> Schedulers could also decide whether or not to allocate multiple long-lived containers
on the same node



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message