Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 924A8200AE8 for ; Tue, 3 May 2016 13:04:14 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 912131609F4; Tue, 3 May 2016 13:04:14 +0200 (CEST) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id BD8881609F3 for ; Tue, 3 May 2016 13:04:13 +0200 (CEST) Received: (qmail 34613 invoked by uid 500); 3 May 2016 11:04:13 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 34604 invoked by uid 99); 3 May 2016 11:04:12 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 May 2016 11:04:12 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id C5D592C0451 for ; Tue, 3 May 2016 11:04:12 +0000 (UTC) Date: Tue, 3 May 2016 11:04:12 +0000 (UTC) From: "Jesus Camacho Rodriguez (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Work started] (HIVE-13677) org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when folding CASE expression MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 03 May 2016 11:04:14 -0000 [ https://issues.apache.org/jira/browse/HIVE-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-13677 started by Jesus Camacho Rodriguez. ------------------------------------------------------ > org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when folding CASE expression > -------------------------------------------------------------------------------------------------------------------- > > Key: HIVE-13677 > URL: https://issues.apache.org/jira/browse/HIVE-13677 > Project: Hive > Issue Type: Bug > Affects Versions: 1.3.0 > Reporter: Jesus Camacho Rodriguez > Assignee: Jesus Camacho Rodriguez > > When folding CASE expressions, children for new expressions are created using the sublist method in List class. Subsequently this leads to problems in Serialization as both list and sublist are modified. In particular, the following Exception is thrown: > {noformat} > org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException > Serialization trace: > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator) > reducer (org.apache.hadoop.hive.ql.plan.ReduceWork) > org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException > Serialization trace: > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) > colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator) > reducer (org.apache.hadoop.hive.ql.plan.ReduceWork) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:1173) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1062) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1076) > at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:445) > at org.apache.hadoop.hive.ql.exec.Utilities.getReduceWork(Utilities.java:319) > at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor$1.call(ReduceRecordProcessor.java:98) > at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75) > at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.(ReduceRecordProcessor.java:95) > at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:136) > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344) > at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181) > at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172) > at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at java.util.ArrayList$SubList.checkForComodification(ArrayList.java:1230) > at java.util.ArrayList$SubList.size(ArrayList.java:1040) > at java.util.AbstractList.add(AbstractList.java:108) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > ... 51 more > {noformat} > To reproduce, the following can be done: > {noformat} > drop table test1; > drop table test2; > create table test1 (id int, desc string) stored as orc; > create table test2 (id int, desc string) stored as orc; > select > case > when (case when a.desc='test' then 1 else 0 end)=0 then 'test' > else null > end as val > FROM test1 a > JOIN test2 b ON a.id=b.id; > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)