hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Baff (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2470) Receiving NPE occasionally on RunningJob.getCounters() call
Date Wed, 11 May 2011 22:21:48 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032129#comment-13032129
] 

Aaron Baff commented on MAPREDUCE-2470:
---------------------------------------

NULL is a reasonable return if it isn't able to get the counters. What if the JobTracker has
the data cached in memory? Couldn't it be possible to return the values from there instead
of the HDFS file? Or would that be a bunch of work to see if it is still in memory?

Just wanted to highlight that this is an NPE at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77),
not just needing to get the Javadocs updated. Should be very quick and easy to simply add
a if( counters == null ) { return null; } before the for-each loop.

> Receiving NPE occasionally on RunningJob.getCounters() call
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-2470
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2470
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.21.0
>         Environment: FreeBSD, Java6, Hadoop r0.21.0
>            Reporter: Aaron Baff
>            Assignee: Robert Joseph Evans
>         Attachments: counters_null_data.pcap
>
>
> This is running in a Java daemon that is used as an interface (Thrift) to get information
and data from MR Jobs. Using JobClient.getJob(JobID) I successfully get a RunningJob object
(I'm checking for NULL), and then rarely I get an NPE when I do RunningJob.getCounters().
This seems to occur after the daemon has been up and running for a while, and in the event
of an Exception, I close the JobClient, set it to NULL, and a new one should then be created
on the next request for data. Yet, I still seem to be unable to fetch the Counters. Below
is the stack trace.
> java.lang.NullPointerException
>             at org.apache.hadoop.mapred.Counters.downgrade(Counters.java:77)
>             at org.apache.hadoop.mapred.JobClient$NetworkedJob.getCounters(JobClient.java:381)
>             at com.telescope.HadoopThrift.service.ServiceImpl.getReportResults(ServiceImpl.java:350)
>             at com.telescope.HadoopThrift.gen.HadoopThrift$Processor$getReportResults.process(HadoopThrift.java:545)
>             at com.telescope.HadoopThrift.gen.HadoopThrift$Processor.process(HadoopThrift.java:421)
>             at org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:697)
>             at org.apache.thrift.server.THsHaServer$Invocation.run(THsHaServer.java:317)
>             at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>             at java.lang.Thread.run(Thread.java:619)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message