hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-15278) PTF+SortMergeJoin = NPE
Date Thu, 24 Nov 2016 00:12:58 GMT
Sergey Shelukhin created HIVE-15278:
---------------------------------------

             Summary: PTF+SortMergeJoin = NPE
                 Key: HIVE-15278
                 URL: https://issues.apache.org/jira/browse/HIVE-15278
             Project: Hive
          Issue Type: Bug
            Reporter: Sergey Shelukhin
            Assignee: Sergey Shelukhin


Manifests as
{noformat}
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115)
	at org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114)
	at org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:340)
	at org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:114)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
	... 29 more
{noformat}

It's actually a somewhat subtle ordering problem in sortmerge - as it stands, it calls different
branches of the tree in closeOp after they themselves have already been closed. Other operators
that clean stuff up in close may result in different errors. The common pattern is
{noformat}
   1125         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
   1126         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
   1127         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:404)
...
   1131         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:428)
   1132         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:388)
   1133         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
...
   1139         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:294)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message