hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5247) NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set to zero.
Date Thu, 19 Feb 2009 05:19:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674895#action_12674895

Vinod K V commented on HADOOP-5247:

Looked at the patch.

One thing. The problem with the client-side exceptions are not the exceptions themselves,
but the lack of information given by them. So we should continue to throw exceptions on the
client but with appropriate error message. For e.g. this code is already in place. (JobClient.java
+1283, runJob())
          if (running == null) {
            throw new IOException("Unable to fetch job status from server.");
We should have similar code inside NetworkedJob.isComplete() and anywhere else JobStatus can
become null. Also the above mentioned code can be moved to the beginning of the try catch
block so that "running" job itself doesn't become null.

Other code comments:
 - trackerToJobsToCleanup doesn't need to be a TreeMap, a HashMap should do.
 - addJobToKill and getJobsToKill don't need to be locked on JT. I think we should protect
all the accesses of trackerToJobsToCleanup by a lock on itself and not on JobTracker.
 - In getJobsToKill, you can combine trackerToJobsToCleanup.get(taskTracker) and trackerToJobsToCleanup.remove(taskTracker)
into a single call Set<JobID> jobs = trackerToJobsToCleanup.remove(taskTracker)
 - The code change for null job in updateTaskStatuses() can be moved further up above the
lookup in taskidToTIPMap.
 - In the testcase, IntSumReducer is duplicated and can be reused from org.apache.hadoop.examples.WordCount
as examples are on the classpath of the junit tests

> NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set
to zero.
> ------------------------------------------------------------------------------------------------
>                 Key: HADOOP-5247
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5247
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Vinod K V
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.20.0
>         Attachments: HADOOP-5247-v1.1.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message