hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3926) Extend the YARN resource model for easier resource-type management and profiles
Date Wed, 22 Jul 2015 00:34:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636068#comment-14636068

Varun Vasudev commented on YARN-3926:

Thanks for the feedback [~kasha]! I'm fine with changing the config variables nomenclature
to the one you suggested. I just wanted to clarify that simply using the same config file
won't avoid the questions you raised(specifically 2 and 3).  The one way we can avoid (2)
and (3) is have versions of the resources configs but I think that's a little complex. We
could mitigate the issue by building some tools to verify that a proposed config file would
work with the existing RM/NM. I'm open to suggestions.

With regards to node labels, I had initial conversations with [~leftnoteasy] but I haven't
thought through the model in enough detail. My initial thinking is that we would modify the
ResourceMapEntry to add a string/list of strings which can be used to specify node labels.

> Extend the YARN resource model for easier resource-type management and profiles
> -------------------------------------------------------------------------------
>                 Key: YARN-3926
>                 URL: https://issues.apache.org/jira/browse/YARN-3926
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Proposal for modifying resource model and profiles.pdf
> Currently, there are efforts to add support for various resource-types such as disk(YARN-2139),
network(YARN-2140), and  HDFS bandwidth(YARN-2681). These efforts all aim to add support for
a new resource type and are fairly involved efforts. In addition, once support is added, it
becomes harder for users to specify the resources they need. All existing jobs have to be
modified, or have to use the minimum allocation.
> This ticket is a proposal to extend the YARN resource model to a more flexible model
which makes it easier to support additional resource-types. It also considers the related
aspect of “resource profiles” which allow users to easily specify the various resources
they need for any given container.

This message was sent by Atlassian JIRA

View raw message