hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navis류승우 <navis....@nexr.com>
Subject Re: random NPE in HiveInputFormat.init() ??
Date Mon, 21 Jul 2014 02:05:52 GMT
I thinks it's fixed in https://issues.apache.org/jira/browse/HIVE-7011,
which will be included in hive-0.14.0. Sadly, there seemed no simple
walkaround for this.

Thanks,
Navis


2014-07-19 15:03 GMT+09:00 Yang <teddyyyy123@gmail.com>:

>
> we are getting a random (happening about 20% of the time, if we repeatedly
> run the same query) error with hive 0.13.0.2
>
>
> java.lang.NullPointerException
>         at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
>         at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:300)
>         at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:520)
>         at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:512)
>         at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
>         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>         at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
>         at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
>         at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
>         at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
>         at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
>         at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>         at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>         at
> org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72)
> Job Submission failed with exception 'java.lang.NullPointerException(null)'
>
>
>
>
>
>
> from google search
> https://www.google.com/search?q=hive+null+pointer+exception+.HiveInputFormat.init(HiveInputFormat.java%3A255)&oq=hive+null+pointer+exception++.HiveInputFormat.init(HiveInputFormat.java%3A255)&aqs=chrome..69i57.16859j0j7&sourceid=chrome&es_sm=93&ie=UTF-8
>
> the top 2 results are pretty relevant:
>
> https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CB8QFjAA&url=http%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fhive-user%2F201302.mbox%2F%253CCAKm%3DR7XyX52-AW5fm5N1vdtW30WQfsw-XAw5%3DmLqn55XeRNSJQ%40mail.gmail.com%253E&ei=FwjKU_rDAdPfoATbsIKgBA&usg=AFQjCNEfg7cyVbTxEl-HeZayFdWfb7Qn_w&sig2=M_0aiHBqrifHIChqiFWXrg&bvm=bv.71198958,d.cGU
>
> this one suggests that it's due to resourcemanager being set to something
> as localhost
>
>
> https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CCkQFjAB&url=http%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fhive-user%2F201308.mbox%2F%253CCAPt%2B2w-0mvzNEk8-9pdpsOag04uG%3D53WF_aRZXG68LsA3MfWGA%40mail.gmail.com%253E&ei=FwjKU_rDAdPfoATbsIKgBA&usg=AFQjCNG4uoRrzD8z1mHy35ztCmt7KRDVOw&sig2=LlreqybSuOurYtbJqRISUQ&bvm=bv.71198958,d.cGU
>
> second one suggests that you have to set the resource manager instead of
> leaving it empty.
>
>
> since we do have a valid server value set to the
> yarn.resourcemanager.address ,  and the above error only appears about 20%
> time, does it mean that our resourcemanager is unstable?
>
> thanks
> Yang
>

Mime
View raw message