hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3590) Null pointer exception in JobTracker when the task tracker is not yet resolved
Date Wed, 18 Jun 2008 04:48:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605825#action_12605825

Amar Kamat commented on HADOOP-3590:

Following are some of the ways this issue can be addressed 
1) Fix {{JobTracker.getParent()}} to return {{null}} in case {{null}} is passed. This means
that the scheduler will select nothing for the _not-yet-resolved_ trackers. The only problem
is we are wasting task-tracker's processing cycles.
2) Throw a IOException with appropriate message like {{"Not Yet Found"}}. Here (1) seems like
a better option and also we are wasting the cycles.
3) Define a new action for the task tracker like {{RETRY}} where the task tracker waits for
some time before resending its old heartbeat. 
4) Randomly give some task rather than wasting cycles.
More? Thoughts? Comments?

> Null pointer exception in JobTracker when the task tracker is not yet resolved
> ------------------------------------------------------------------------------
>                 Key: HADOOP-3590
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3590
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.18.0
> 2008-06-17 13:29:56,711 INFO org.apache.hadoop.mapred.JobTracker: Got heartbeat from:
xxx (initialContact: false acceptNewTasks: true) with responseId: 0
> 2008-06-17 13:29:56,711 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 123,
call heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@123, true, true, -1) from xxx error:
java.io.IOException: java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
>        at org.apache.hadoop.mapred.JobTracker.getParentNode(JobTracker.java:1327)
>        at org.apache.hadoop.mapred.JobInProgress.findNewMapTask(JobInProgress.java:1142)
>        at org.apache.hadoop.mapred.JobInProgress.obtainNewMapTask(JobInProgress.java:685)
>        at org.apache.hadoop.mapred.JobTracker.getNewTaskForTaskTracker(JobTracker.java:1708)
>        at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:1431)
>        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> ----
> Looks like a corner case. This happens when the task tracker just joins the JT and asks
for a task to run. In a case where the task tracker is not yet resolved, this can lead to
null pointer exception. Remember that the task trackers are  added to separate queue and resolved
by a separate thread i.e there is no forced resolution for task trackers. There is no side
effect of this bug since the task tracker will try again and also the job runs to completion.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message