hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4309) Make locatlity in YARN's container assignment and task scheduling pluggable for other deployment topology
Date Fri, 13 Jul 2012 06:47:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413526#comment-13413526
] 

Bikas Saha commented on MAPREDUCE-4309:
---------------------------------------

1) AppSchedulingInfo.java
In allocateNodeGroupLocal(...), why not call decrementOutstanding(requests.get(priority).get(RMNode.ANY))
instead of the following
{code}
    // Do not remove ANY
    ResourceRequest offSwitchRequest = requests.get(priority).get(
        RMNode.ANY);
    offSwitchRequest.setNumContainers(offSwitchRequest.getNumContainers() 
{code}

2) Many methods/variables of multilple classes (LeafQueue/FifoScheduler etc) have been made
protected but I dont see them subclassed anywhere. Whats the purpose of those changes?

3) Some objects are not being created via reflection and config. Its not clear what the purpose
of those are?
                
> Make locatlity in YARN's container assignment and task scheduling pluggable for other
deployment topology
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4309
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4309
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0-alpha
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HADOOP-8474-ContainerAssignmentTaskScheduling-pluggable.patch, MAPREDUCE-4309-v2.patch,
MAPREDUCE-4309-v3.patch, MAPREDUCE-4309-v4.patch, MAPREDUCE-4309.patch
>
>
> There are several classes in YARN’s container assignment and task scheduling algorithms
that relate to data locality which were updated to give preference to running a container
on other locality besides node-local and rack-local (like nodegroup-local). This propose to
make these data structure/algorithms pluggable, like: SchedulerNode, RMNodeImpl, etc. The
inner class ScheduledRequests was made a package level class to it would be easier to create
a subclass, ScheduledRequestsWithNodeGroup.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message