hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Pivovarov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7738) tez select sum(decimal) from union all of decimal and null throws NPE
Date Fri, 15 Aug 2014 03:19:18 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098092#comment-14098092
] 

Alexander Pivovarov commented on HIVE-7738:
-------------------------------------------

Problem does not exist if cast to double instead of to decimal
select sum(a) from (
  select cast(1.1 as double) a from dual
  union all
  select cast(null as double) a from dual
) t;

> tez select sum(decimal) from union all of decimal and null throws NPE
> ---------------------------------------------------------------------
>
>                 Key: HIVE-7738
>                 URL: https://issues.apache.org/jira/browse/HIVE-7738
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 0.13.1
>            Reporter: Alexander Pivovarov
>
> if run this query using tez engine then hive will throw NPE
> select sum(a) from (
>   select cast(1.1 as decimal) a from dual
>   union all
>   select cast(null as decimal) a from dual
> ) t;
> hive> select sum(a) from (
>     >   select cast(1.1 as decimal) a from dual
>     >   union all
>     >   select cast(null as decimal) a from dual
>     > ) t;
> Query ID = apivovarov_20140814200909_438385b2-4147-47bc-98a0-a01567bbb5c5
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: application_1407388228332_5616)
> Map 1: -/-	Map 4: -/-	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 0/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 0/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 1/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 1/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 1/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 1/1	Reducer 3: 0/1	
> Map 1: 0/1	Map 4: 1/1	Reducer 3: 0/1	
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1407388228332_5616_1_02, diagnostics=[Task
failed, taskId=task_1407388228332_5616_1_02_000000, diagnostics=[AttemptID:attempt_1407388228332_5616_1_02_000000_0
Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization
failed
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> 	at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> 	... 6 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> 	... 7 more
> Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_1
Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization
failed
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> 	at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> 	... 6 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> 	... 7 more
> Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_2
Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization
failed
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> 	at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> 	... 6 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> 	... 7 more
> Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_3
Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization
failed
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
> 	at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164)
> 	... 6 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> 	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
> 	... 7 more
> ], Vertex failed as one or more tasks failed. failedTasks:1]
> Vertex killed, vertexName=Reducer 3, vertexId=vertex_1407388228332_5616_1_01, diagnostics=[Vertex
received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0]
> DAG failed due to vertex failure. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message