hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-450) Define value for * in the scheduling protocol
Date Wed, 06 Mar 2013 20:38:13 GMT

     [ https://issues.apache.org/jira/browse/YARN-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhijie Shen updated YARN-450:

    Attachment: YARN-450_1.patch

I've created a patch that defines ANY = "*" in RMNode and make all classes refer to this constant.
All the modified test cases were passed locally.

However, I've the following two concerns:

1. I need to modify pom.xml of hadoop-yarn-client and hadoop-mapreduce-client-app to add hadoop-yarn-server-resourcemanager
into their build dependency, because there're classes in these two sub-projects who refer
to RMNode. I'm a bit worried about the potential impact on jar isolation.

Another choice is to define ANY = "*" in ResourceRequest, which is located in hadooop-yarn-api,
i.e., the existing dependency of the two aforementioned sub-projects. Semantically, it is
not natural to do this, because ANY means an arbitrary host name, which should be the property
of a node.

2. There's another Enum class called Locality, which define the three types of node locality:
NODE_LOCAL, RACK_LOCAL, OFF_SWITCH. Here ANY indicates the node is not necessary to be host/rack
local. Do we need to make the names consistent, i.e., use OFF_SWITCH (or OFF_SWITCH_HOST)
instead of ANY?
> Define value for * in the scheduling protocol
> ---------------------------------------------
>                 Key: YARN-450
>                 URL: https://issues.apache.org/jira/browse/YARN-450
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Zhijie Shen
>         Attachments: YARN-450_1.patch
> The ResourceRequest has a string field to specify node/rack locations. For the cross-rack/cluster-wide
location (ie when there is no locality constraint) the "*" string is used everywhere. However,
its not defined anywhere and each piece of code either defines a local constant or uses the
string literal. Defining "*" in the protocol and removing other local references from the
code base will be good.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message