hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2237) Lost heartbeat response containing MapTask throws NPE when it is resent
Date Tue, 04 Jan 2011 07:49:48 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977156#action_12977156
] 

Arun C Murthy commented on MAPREDUCE-2237:
------------------------------------------

Yes, the split was set to null to save memory. The fix should be trivial - add the split info
to all LaunchTaskAction objects in the HeartbeatResponse object.

An important task is to check why the heartbeat response was lost. That problem was mainly
due to our RPC system, something we haven't seen in a long while...

> Lost heartbeat response containing MapTask throws NPE when it is resent
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2237
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2237
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.2, 0.21.0, 0.22.0
>            Reporter: Todd Lipcon
>
> When the JT sends a heartbeat response, it records it in trackerToHeartbeatResponseMap.
But after MapTask writes its input split, it sets that split to null (assumedly to save memory?).
So, if the heartbeat response is lost, and the JT needs to resend it, it will throw NPE since
the split information has been lost.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message