hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1678) NPE in MapJoin
Date Fri, 01 Oct 2010 05:45:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916789#action_12916789
] 

Amareshwari Sriramadasu commented on HIVE-1678:
-----------------------------------------------

The following query :
select /*+MAPJOIN(src, myinput1) */ count(srcpart.key) from srcpart join src on (srcpart.value=src.value)
join myinput1 on (srcpart.key=myinput1.key);
launches three MapReduce jobs in which third one fails with NPE.

The task logs : 
{noformat}
2010-10-01 11:07:02,324 INFO ExecMapper: 
<MAP>Id =11
  <Children>
    <SEL>Id =0
      <Children>
        <MAPJOIN>Id =1
          <Children>
            <FS>Id =2
              <Parent>Id = 1 null<\Parent>
            <\FS>
          <\Children>
          <Parent>Id = 0 nullId = 3 
        <TS>Id =3
          <Children>null
          <\Children>
        <\TS><\Parent>
        <\MAPJOIN>
      <\Children>
      <Parent>Id = 11 null<\Parent>
    <\SEL>
  <\Children>
<\MAP>
2010-10-01 11:07:02,324 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initializing Self
11 MAP
2010-10-01 11:07:02,324 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing Self
0 SEL
2010-10-01 11:07:02,327 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT struct<_col0:int>
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Operator 0 SEL
initialized
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing children
of 0 SEL
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.MapJoinOperator: Initializing
child 1 MAPJOIN
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initialization
Done 0 SEL
2010-10-01 11:07:02,332 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done
11 MAP
2010-10-01 11:07:02,334 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 11 forwarding 1 rows
2010-10-01 11:07:02,334 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 0 forwarding 1
rows
2010-10-01 11:07:02,338 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row {"_col0":238}
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:483)
	at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:329)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:219)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1021)
	at org.apache.hadoop.mapred.Child.main(Child.java:213)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:177)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:459)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:710)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:459)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:710)
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:464)
	... 9 more
{noformat}

>From the logs we can see that MapJoinOperator is not intialized at all.

> NPE in MapJoin 
> ---------------
>
>                 Key: HIVE-1678
>                 URL: https://issues.apache.org/jira/browse/HIVE-1678
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Amareshwari Sriramadasu
>
> The query with two map joins and a group by fails with following NPE:
> Caused by: java.lang.NullPointerException
>         at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:177)
>         at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
>         at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
>         at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:464)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message