hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Iyappan Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-746) When a task tracker is killed, there is a Null Pointer exception thrown.
Date Fri, 10 Jul 2009 08:23:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729570#action_12729570
] 

Iyappan Srinivasan commented on MAPREDUCE-746:
----------------------------------------------

The job tracker logs just before this null pointer exception. I have replaced teh anme of
killed task tracker server name with this string "<killed TT>"

"2009-07-10 07:51:43,626 INFO org.apache.hadoop.mapred.JobTracker: Adding <killed TT>
to the blacklist across all jobs
2009-07-10 07:51:43,628 INFO org.apache.hadoop.mapred.JobTracker: Blacklisting tracker : <killed
TT>  Reason for blacklisting is : EXCEEDING_FAILURES
2009-07-10 07:51:43,631 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9020, call
heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@1fb050c, false, false, true, 269) from
98.136.99.54:55815: error: java.io.IOException: java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException"


> When a  task tracker is killed, there is a Null Pointer exception thrown.
> -------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-746
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-746
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>            Reporter: Iyappan Srinivasan
>            Priority: Minor
>
> When a task tracker is killed, the job completes. But tehre is a null pointer exception
thrown:
> java.io.IOException: java.lang.NullPointerException
> 	at org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.removeHostCapacity(JobTracker.java:759)
> 	at org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.blackListTracker(JobTracker.java:624)
> 	at org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.incrementFaults(JobTracker.java:601)
> 	at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:2337)
> 	at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2998)
> 	at org.apache.hadoop.mapred.JobInProgress.jobComplete(JobInProgress.java:2584)
> 	at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2473)
> 	at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:1047)
> 	at org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:3867)
> 	at org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3079)
> 	at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2817)
> 	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:960)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:958)
> Steps to reproduce the issue:
> 1) Bring up a 5 node cluster.
> 2) set mapred.max.tracker.failures to 1
> 3) Run a sleep command with 5 maps and 5 redcues.
> 4) Kill a task tracker, when map is 80% complete.
> 5) Kill the task tracker, by using "kill -9 process_id". 
> 6) At the time of killing, it was running 2 maps and 2 reduces.
> 7) Allow 12 minutes to elapse for that task tracker to go out of job nodes list
> 8) Then job completes successfully by giving the task attempts of the lost trackers to
other nodes.
> 9) The job tracker logs has this exception.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message