Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 95CCA1785C for ; Mon, 6 Oct 2014 22:26:35 +0000 (UTC) Received: (qmail 22744 invoked by uid 500); 6 Oct 2014 22:26:35 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 22675 invoked by uid 500); 6 Oct 2014 22:26:35 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 22655 invoked by uid 500); 6 Oct 2014 22:26:34 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 22652 invoked by uid 99); 6 Oct 2014 22:26:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Oct 2014 22:26:34 +0000 Date: Mon, 6 Oct 2014 22:26:34 +0000 (UTC) From: "Brock Noland (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5865: ------------------------------- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Ben, thank you so much for your contribution! I have committed this to trunk and made you a contributor so you can assign yourself JIRAs! > AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' > ---------------------------------------------------------------------------------------------------- > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug > Affects Versions: 0.11.0, 0.12.0, 0.13.0 > Reporter: Ben Roling > Assignee: Ben Roling > Fix For: 0.15.0 > > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines "avro.java.string"="String", this assumption does not hold, resulting in a ClassCastException. > I think a simple fix would be to define 'mapDatum' with type Map instead of Map. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the "avro.java.string" attribute on my "map" type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)