hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-50) NPE in heartbeat when the configured topology script doesn't exist
Date Thu, 09 Feb 2012 17:27:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204667#comment-13204667

Steve Loughran commented on MAPREDUCE-50:

I've seen logs in emails that indicate this is still possible to arise in the 0.20.20x series.

The problem is that the {{ScriptBasedMapping.RawScriptMapping.resolve()} method behaves as
 # return "/default-rack" if there is no script
 # return null if the script fails
 # {{CachedDNSToSwitchMapping.resolve()} (which invokes this inner resolve) sees the null
pointer, doesn't add those entries to its cache, then builds a list of results for each supplied
argument -and for unknown hosts returns null.
 # The JobTracker doesn't expect null.

What to do?
 * The WARN could somehow be raised to a higher level so that it doesn't get lost to the user
-or the error message changed to be more meaningful to users who don't want to delve into
the source. 
 * The JT (and its YARN successor) need a policy of handling null-ness.
 * Better preflight checking of the shell script

Unit tests could verify how the Namenode, JT and RM handle null entries with a switch mapping
that returns null always; trivial.
> NPE in heartbeat when the configured topology script doesn't exist
> ------------------------------------------------------------------
>                 Key: MAPREDUCE-50
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-50
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message