hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6922) NullPointerException in collect_set() UDAF
Date Fri, 18 Apr 2014 03:01:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973738#comment-13973738
] 

Xuefu Zhang commented on HIVE-6922:
-----------------------------------

Sounds good then. Thanks for the explanation.

> NullPointerException in collect_set() UDAF
> ------------------------------------------
>
>                 Key: HIVE-6922
>                 URL: https://issues.apache.org/jira/browse/HIVE-6922
>             Project: Hive
>          Issue Type: Bug
>          Components: UDF
>            Reporter: Sun Rui
>            Assignee: Sun Rui
>         Attachments: HIVE-6922.patch
>
>
> Steps to reproduce the bug:
> {noformat}
> create table temp(key int, value string);
> -- leave the table empty
> select collect_set(key) from temp where key=0;
> Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:326)
> 	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:471)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
> 	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:577)
> 	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:318)
> 	... 7 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.merge(GenericUDAFMkCollectionEvaluator.java:140)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:186)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1132)
> 	... 9 more
> {noformat}
> The root cause is that in GenericUDAFMkCollectionEvaluator.merge() partialResult could
be null but is not validated before it is used.
> {code}
>     List<Object> partialResult = (ArrayList<Object>) internalMergeOI.getList(partial);
>     for(Object i : partialResult) {
>       putIntoCollection(i, myagg);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message