hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Reopened: (HIVE-1422) skip counter update when RunningJob.getCounters() returns null
Date Sat, 24 Jul 2010 20:37:49 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Joydeep Sen Sarma reopened HIVE-1422:

reopening. i think the bug still exists:

jc.getJob(rj.getJobID()) can return null if the jobtracker has retired the job by the time
we poll for it. this happens very rarely. we are setting the returned value in the execdriver
handle. and then trying to dereference it later on.

if we have a valid runningjob reference - i don't think we have seen the getCounters call
return null (in the past).

> skip counter update when RunningJob.getCounters() returns null
> --------------------------------------------------------------
>                 Key: HIVE-1422
>                 URL: https://issues.apache.org/jira/browse/HIVE-1422
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: John Sichi
>            Assignee: John Sichi
>             Fix For: 0.7.0
>         Attachments: HIVE-1422.1.patch
> Under heavy load circumstances on some Hadoop versions, we may get a NPE from trying
to dereference a null Counters object.  I don't have a unit test which can reproduce it, but
here's an example stack from a production cluster we saw today:
> 10/06/21 13:01:10 ERROR exec.ExecDriver: Ended Job = job_201005200457_701060 with exception
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:999)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:503)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:390)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:697)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message