hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
Date Tue, 06 Mar 2012 10:04:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223131#comment-13223131

Steve Loughran commented on MAPREDUCE-1740:

Looks OK on a quick review; will need to test a bit more.

(trunk only) {{AbstractDNSToSwitchMapping()}} has a {{getSwitchMap()}} method which can return
the map of host->switch (or null in the base class), which could be used for more assertions.
This class isn't in branch1, so these asserts will have to be left out there.

It also has {{dumpTopology()}} method to convert the map to a string, which can be used for
diagnostics in logs or assertions.

style issues
 # please follow hadoop project spacing rules esp. in if () conditions
 # use LOG over {{System.out}} even in tests
 # some of the static fields used in the tests could be made final
> NPE in getMatchingLevelForNodes when node locations are variable depth
> ----------------------------------------------------------------------
>                 Key: MAPREDUCE-1740
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.3
>            Reporter: Todd Lipcon
>         Attachments: MAPREDUCE-1740.patch, mapreduce-1740.txt
> In getMatchingLevelForNodes, we assume that both nodes have the same "depth" (ie number
of path components). If the user provides a topology script that assigns one node a path like
/foo/bar/baz and another node a path like /foo/blah, this function will throw an NPE.
> I'm not sure if there are other places where we assume that all node locations have a
constant number of paths. If so we should check the output of the topology script aggressively
to be sure this is the case. Otherwise I think we simply need to add && n2 != null
to the while loop

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message