hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4893) MR AppMaster can do sub-optimal assignment of containers to map tasks leading to poor node locality
Date Wed, 09 Jan 2013 15:32:13 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated MAPREDUCE-4893:
-----------------------------------------------

    Status: Open  (was: Patch Available)

Can you please upmerge the patch to latest code, the test isn't applying anymore at all.

Patch looks good overall, some minor nits:
 - +792 after patch: The condition should be a simple else: {{ else if (!PRIORITY_MAP.equals(priority))
{ }}
 - +1024 after patch is dead code:         LinkedList<TaskAttemptId> list = mapsHostMapping.get(host);

Will review test changes after your update.
                
> MR AppMaster can do sub-optimal assignment of containers to map tasks leading to poor
node locality
> ---------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4893
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4893
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>             Fix For: 3.0.0
>
>         Attachments: MAPREDUCE-4893.1.patch
>
>
> Say the MR AppMaster asks the RM for 3 containers on nodes n1, n2 and n3. There are 10
node n1-n10 in the same rack. The RM can give it allocated containers in the list order n5,
n2, n1. The way AM map->container assignment happens, the AM will try to assign node local
maps to n5, failing which it will assign rack local maps to n5. These rack local maps could
be node local on n2 and n1 and would have been assigned to containers on n1 and n2 if the
AM had not made an early rack local match for them on n5. This can lead to poor locality.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message