Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4406710FBF for ; Tue, 3 Sep 2013 20:41:52 +0000 (UTC) Received: (qmail 79936 invoked by uid 500); 3 Sep 2013 20:41:51 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 79881 invoked by uid 500); 3 Sep 2013 20:41:51 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 79872 invoked by uid 500); 3 Sep 2013 20:41:51 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 79869 invoked by uid 99); 3 Sep 2013 20:41:51 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Sep 2013 20:41:51 +0000 Date: Tue, 3 Sep 2013 20:41:51 +0000 (UTC) From: "Hari Sankar Sivarama Subramaniyan (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HIVE-5199) Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Hari Sankar Sivarama Subramaniyan created HIVE-5199: ------------------------------------------------------- Summary: Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3 Key: HIVE-5199 URL: https://issues.apache.org/jira/browse/HIVE-5199 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Priority: Critical Custom serdes which used to work in HDP 1.1 is no longer working with HDP 1.3. The issue happens when the partition serde is not of settable type in HDP 1.3. The below exception happens via FetchOperator as well as MapOperator. Inside FetchOperator consider the following call: getRecordReader()->ObjectInspectorConverters. getConverter() The output object inspector is of settable type(because it is generated via ObjectInspectorConverters.getConvertedOI()) where as the input object inspector that gets passed as serde.getObjectorInspector() and is non-settable. Inside getConverter(), the (inputOI.equals(outputOI)) check fails and the switch statement tries to cast the non-settable object inspector to a settable object inspector. The stack trace as follows: 2013-08-28 17:57:25,307 ERROR CliDriver (SessionState.java:printError(432)) - Failed with exception java.io.IOException:java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector java.io.IOException: java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:160) Caused by: java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.(ObjectInspectorConverters.java:307) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:406) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira