hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5623) TestJobCleanup fails because of RejectedExecutionException and NPE.
Date Fri, 22 Nov 2013 12:49:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13829932#comment-13829932

Tsuyoshi OZAWA commented on MAPREDUCE-5623:

Thank you for the comment, Jason. 

    MRClientService#getCounters can access it, but only via a reference to a JobImpl object
which will have jobCounters initialized to a non-null value by the time the getAllCounters()
method is invoked.

I thought as follows when I created patch:
1. Java's loaders' threads initializes jobCounters. Initialized jobCounters can be seen from
a thread which initializes jobCounters.
2. MRClientService#getCounters is called, but it's not a initialization thread. Therefore,
it can return null and the RPC returns null.

Your point is that 1. of my thought is wrong and initialized values are barriered after creating
its instances. I cannot say which is correct because I cannot find documents about initialization
sequence of Java instances. My thought was wrong?

Or, I've noticed that Job#getCounters() can return null in some cases:
   * Gets the counters for this job. May return null if the job has been
   * retired and the job is no longer in the completed job store.
   * @return the counters for this job.
   * @throws IOException
  public Counters getCounters() 
This can be reason that job.getCounters returns null.

> TestJobCleanup fails because of RejectedExecutionException and NPE.
> -------------------------------------------------------------------
>                 Key: MAPREDUCE-5623
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5623
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: MAPREDUCE-5623.1.patch
> org.apache.hadoop.mapred.TestJobCleanup can fail because of RejectedExecutionException
by NonAggregatingLogHandler. This problem is described in YARN-1409. TestJobCleanup can still
fail after fixing RejectedExecutionException, because of NPE by Job#getCounters()'s returning
> {code}
> -------------------------------------------------------------------------------
> Test set: org.apache.hadoop.mapred.TestJobCleanup
> -------------------------------------------------------------------------------
> Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 140.933 sec <<<
FAILURE! - in org.apache.hadoop.mapred.TestJobCleanup
> testCustomAbort(org.apache.hadoop.mapred.TestJobCleanup)  Time elapsed: 31.068 sec  <<<
> java.lang.NullPointerException: null
>         at org.apache.hadoop.mapred.TestJobCleanup.testFailedJob(TestJobCleanup.java:199)
>         at org.apache.hadoop.mapred.TestJobCleanup.testCustomAbort(TestJobCleanup.java:296)
> {code}

This message was sent by Atlassian JIRA

View raw message