Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C5B5B200C47 for ; Thu, 16 Mar 2017 00:19:45 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C441A160B7C; Wed, 15 Mar 2017 23:19:45 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 182C4160B78 for ; Thu, 16 Mar 2017 00:19:44 +0100 (CET) Received: (qmail 73872 invoked by uid 500); 15 Mar 2017 23:19:43 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 73860 invoked by uid 99); 15 Mar 2017 23:19:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Mar 2017 23:19:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 2A513C161C for ; Wed, 15 Mar 2017 23:19:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.651 X-Spam-Level: X-Spam-Status: No, score=0.651 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id G71oE16CM3ja for ; Wed, 15 Mar 2017 23:19:42 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 5D5A35F4AA for ; Wed, 15 Mar 2017 23:19:42 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D9FB1E026E for ; Wed, 15 Mar 2017 23:19:41 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 95865243A8 for ; Wed, 15 Mar 2017 23:19:41 +0000 (UTC) Date: Wed, 15 Mar 2017 23:19:41 +0000 (UTC) From: "Jerry He (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-16247) SparkSQL Avro serialization doesn't handle enums correctly MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 15 Mar 2017 23:19:46 -0000 [ https://issues.apache.org/jira/browse/HBASE-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927153#comment-15927153 ] Jerry He commented on HBASE-16247: ---------------------------------- Hi, [~busbey] Can you provide more details on the problem with hbase spark sql handling of the Avro enum type? > SparkSQL Avro serialization doesn't handle enums correctly > ---------------------------------------------------------- > > Key: HBASE-16247 > URL: https://issues.apache.org/jira/browse/HBASE-16247 > Project: HBase > Issue Type: Bug > Components: spark > Affects Versions: 2.0.0 > Reporter: Sean Busbey > Priority: Critical > Fix For: 2.0.0 > > > Avro's generic api expects GenericEnumSymbol as the runtime type for instances of fields that are of Avro type ENUM. The Avro 1.7 libraries are lax in some cases for handling this, but the 1.8 libraries are strict. We should proactively fix our serialization. > (the lax serialization in 1.7 fails for some nested use in unions, see AVRO-997 for details) -- This message was sent by Atlassian JIRA (v6.3.15#6346)