hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
Date Thu, 09 Sep 2010 22:43:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907808#action_12907808
] 

Todd Lipcon commented on MAPREDUCE-1740:
----------------------------------------

Attached patch fixes the NPE, though I haven't had a chance to test on an actual misconfigured
cluster.

Is this the right approach or should we just do a stricter job of verifying the results of
the topology mapping script to ensure that all hosts are the same level? Does anyone have
a legitimate use case for locality of varying depths?

> NPE in getMatchingLevelForNodes when node locations are variable depth
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1740
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.3, 0.21.0, 0.22.0
>            Reporter: Todd Lipcon
>         Attachments: mapreduce-1740.txt
>
>
> In getMatchingLevelForNodes, we assume that both nodes have the same "depth" (ie number
of path components). If the user provides a topology script that assigns one node a path like
/foo/bar/baz and another node a path like /foo/blah, this function will throw an NPE.
> I'm not sure if there are other places where we assume that all node locations have a
constant number of paths. If so we should check the output of the topology script aggressively
to be sure this is the case. Otherwise I think we simply need to add && n2 != null
to the while loop

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message