hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
Date Fri, 20 May 2011 13:22:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036811#comment-13036811
] 

Robert Joseph Evans commented on MAPREDUCE-2470:
------------------------------------------------

The difference is that with Job.getStatus it calls internally updateStatus which will throw
an IOException if it is unable to get an updated status from the JobTracker.  The only time
you can get an NPE from that is if you catch the IOException, ignore it, and keep trying to
use the Job object what was originally created.  We can fix that, by modifying updateStatus
to not overwrite Job.status until it knows that the updated status is not null.  But that
was not explicitly part of this JIRA, and I could not find any other JIRA to cover it so I
though it was not a big deal, and most likely worked as designed. 

> Receiving NPE occasionally on RunningJob.getCounters() call
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-2470
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.21.0
>         Environment: FreeBSD, Java6, Hadoop r0.21.0
>            Reporter: Aaron Baff
>            Assignee: Robert Joseph Evans
>         Attachments: MAPREDUCE-2470-v1.patch, MAPREDUCE-2470-v2.patch, counters_null_data.pcap
>
>
> This is running in a Java daemon that is used as an interface (Thrift) to get information
and data from MR Jobs. Using JobClient.getJob(JobID) I successfully get a RunningJob object
(I'm checking for NULL), and then rarely I get an NPE when I do RunningJob.getCounters().
This seems to occur after the daemon has been up and running for a while, and in the event
of an Exception, I close the JobClient, set it to NULL, and a new one should then be created
on the next request for data. Yet, I still seem to be unable to fetch the Counters. Below
is the stack trace.
> java.lang.NullPointerException
>             at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77)
>             at org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381)
>             at com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350)
>             at com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545)
>             at com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421)
>             at org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697)
>             at org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317)
>             at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>             at java.lang.Thread.run(Thread.java:619)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message