hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
Date Tue, 20 Mar 2012 18:41:38 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233658#comment-13233658
] 

Arun C Murthy commented on MAPREDUCE-1740:
------------------------------------------

This is nice to fix. However, we should look at HDFS too. Doing it in isolation in MR isn't
sufficient.
                
> NPE in getMatchingLevelForNodes when node locations are variable depth
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1740
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.3
>            Reporter: Todd Lipcon
>            Assignee: Ahmed Radwan
>         Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, MAPREDUCE-1740_trunk.patch,
mapreduce-1740.txt
>
>
> In getMatchingLevelForNodes, we assume that both nodes have the same "depth" (ie number
of path components). If the user provides a topology script that assigns one node a path like
/foo/bar/baz and another node a path like /foo/blah, this function will throw an NPE.
> I'm not sure if there are other places where we assume that all node locations have a
constant number of paths. If so we should check the output of the topology script aggressively
to be sure this is the case. Otherwise I think we simply need to add && n2 != null
to the while loop

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message