hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-3234) Locality scheduling broken due to mismatch between IPs and hosts
Date Thu, 20 Oct 2011 18:12:12 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mahadev konar resolved MAPREDUCE-3234.

    Resolution: Not A Problem
> Locality scheduling broken due to mismatch between IPs and hosts
> ----------------------------------------------------------------
>                 Key: MAPREDUCE-3234
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3234
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Priority: Blocker
>         Attachments: MR-3234.1.patch
> I noticed that, on a single-rack cluster, I wasn't getting hardly any data locality.
The issue appears to be the code in RMContainerAllocator which changes the resource requests
to use IP addresses instead of hostnames:
> {code}
>            //host comes from data splitLocations which are hostnames. Containers
>            // use IP addresses.
>            //TODO Temporary fix for locality. Use resolvers from h-common. 
>            // Cache to make this more efficient ?
> {code}
> However, at least on my cluster, the resource manager sees node resources as hostnames,
not IPs. Removing this code fixed data locality.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message