hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-450) Define value for * in the scheduling protocol
Date Wed, 06 Mar 2013 20:38:13 GMT

     [ https://issues.apache.org/jira/browse/YARN-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhijie Shen updated YARN-450:
-----------------------------

    Attachment: YARN-450_1.patch

I've created a patch that defines ANY = "*" in RMNode and make all classes refer to this constant.
All the modified test cases were passed locally.

However, I've the following two concerns:

1. I need to modify pom.xml of hadoop-yarn-client and hadoop-mapreduce-client-app to add hadoop-yarn-server-resourcemanager
into their build dependency, because there're classes in these two sub-projects who refer
to RMNode. I'm a bit worried about the potential impact on jar isolation.

Another choice is to define ANY = "*" in ResourceRequest, which is located in hadooop-yarn-api,
i.e., the existing dependency of the two aforementioned sub-projects. Semantically, it is
not natural to do this, because ANY means an arbitrary host name, which should be the property
of a node.

2. There's another Enum class called Locality, which define the three types of node locality:
NODE_LOCAL, RACK_LOCAL, OFF_SWITCH. Here ANY indicates the node is not necessary to be host/rack
local. Do we need to make the names consistent, i.e., use OFF_SWITCH (or OFF_SWITCH_HOST)
instead of ANY?
                
> Define value for * in the scheduling protocol
> ---------------------------------------------
>
>                 Key: YARN-450
>                 URL: https://issues.apache.org/jira/browse/YARN-450
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Zhijie Shen
>         Attachments: YARN-450_1.patch
>
>
> The ResourceRequest has a string field to specify node/rack locations. For the cross-rack/cluster-wide
location (ie when there is no locality constraint) the "*" string is used everywhere. However,
its not defined anywhere and each piece of code either defines a local constant or uses the
string literal. Defining "*" in the protocol and removing other local references from the
code base will be good.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message