Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D324C11330 for ; Fri, 15 Aug 2014 07:07:18 +0000 (UTC) Received: (qmail 31929 invoked by uid 500); 15 Aug 2014 07:07:18 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 31851 invoked by uid 500); 15 Aug 2014 07:07:18 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 31758 invoked by uid 500); 15 Aug 2014 07:07:18 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 31752 invoked by uid 99); 15 Aug 2014 07:07:18 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Aug 2014 07:07:18 +0000 Date: Fri, 15 Aug 2014 07:07:18 +0000 (UTC) From: "Alexander Pivovarov (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-7738) tez select sum(decimal) from union all of decimal and null throws NPE MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-7738: -------------------------------------- Component/s: (was: Tez) Serializers/Deserializers > tez select sum(decimal) from union all of decimal and null throws NPE > --------------------------------------------------------------------- > > Key: HIVE-7738 > URL: https://issues.apache.org/jira/browse/HIVE-7738 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers > Affects Versions: 0.13.1 > Reporter: Alexander Pivovarov > Attachments: HIVE-7738.patch > > > if run this query using tez engine then hive will throw NPE > {code} > select sum(a) from ( > select cast(1.1 as decimal) a from dual > union all > select cast(null as decimal) a from dual > ) t; > {code} > hive> select sum(a) from ( > > select cast(1.1 as decimal) a from dual > > union all > > select cast(null as decimal) a from dual > > ) t; > Query ID = apivovarov_20140814200909_438385b2-4147-47bc-98a0-a01567bbb5c5 > Total jobs = 1 > Launching Job 1 out of 1 > Status: Running (application id: application_1407388228332_5616) > Map 1: -/- Map 4: -/- Reducer 3: 0/1 > Map 1: 0/1 Map 4: 0/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 0/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1 > Map 1: 0/1 Map 4: 1/1 Reducer 3: 0/1 > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1407388228332_5616_1_02, diagnostics=[Task failed, taskId=task_1407388228332_5616_1_02_000000, diagnostics=[AttemptID:attempt_1407388228332_5616_1_02_000000_0 Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188) > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307) > at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553) > Caused by: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145) > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164) > ... 6 more > Caused by: java.lang.NullPointerException > at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61) > at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106) > at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121) > ... 7 more > Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_1 Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188) > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307) > at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553) > Caused by: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145) > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164) > ... 6 more > Caused by: java.lang.NullPointerException > at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61) > at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106) > at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121) > ... 7 more > Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_2 Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188) > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307) > at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553) > Caused by: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145) > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164) > ... 6 more > Caused by: java.lang.NullPointerException > at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61) > at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106) > at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121) > ... 7 more > Container released by application, AttemptID:attempt_1407388228332_5616_1_02_000000_3 Info:Error: java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:188) > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307) > at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553) > Caused by: java.lang.RuntimeException: Map operator initialization failed > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145) > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:164) > ... 6 more > Caused by: java.lang.NullPointerException > at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableConstantHiveDecimalObjectInspector.precision(WritableConstantHiveDecimalObjectInspector.java:61) > at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFSum$GenericUDAFSumHiveDecimal.init(GenericUDAFSum.java:106) > at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:362) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:67) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460) > at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425) > at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) > at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121) > ... 7 more > ], Vertex failed as one or more tasks failed. failedTasks:1] > Vertex killed, vertexName=Reducer 3, vertexId=vertex_1407388228332_5616_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0] > DAG failed due to vertex failure. failedVertices:1 killedVertices:1 > FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask -- This message was sent by Atlassian JIRA (v6.2#6252)