Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0F02C200D37 for ; Thu, 26 Oct 2017 01:32:09 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0D805160BE0; Wed, 25 Oct 2017 23:32:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E84C6160BDA for ; Thu, 26 Oct 2017 01:32:07 +0200 (CEST) Received: (qmail 18577 invoked by uid 500); 25 Oct 2017 23:32:06 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 18498 invoked by uid 99); 25 Oct 2017 23:32:06 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Oct 2017 23:32:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 82142C474C for ; Wed, 25 Oct 2017 23:32:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id kcDGg_-zQ5h4 for ; Wed, 25 Oct 2017 23:32:04 +0000 (UTC) Received: from mail-oi0-f54.google.com (mail-oi0-f54.google.com [209.85.218.54]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 56B675FC1C for ; Wed, 25 Oct 2017 23:32:04 +0000 (UTC) Received: by mail-oi0-f54.google.com with SMTP id m198so2835491oig.5 for ; Wed, 25 Oct 2017 16:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Z2M+gqqaQpn9FcBPJlH4AznMxN1PloeM6UtHP44geQY=; b=vWYkWOGjGdnfmAoWx0bQpajQQacHo0UgQXZoRgR7NrM3herXHBl7gGzfl+2Xg1Jn4/ gOrrc8+HICNuhj7pGLAu7Hd9FaayXZDqbSlYWWaJbXxCswomWW/+8219CwUoa7iU7gB/ bIWg1oxqZy94BP11ti0oRLbJh5ekvsJocPd/pjn75TSG2nEE+W+sJtJFoXQFeoNm8ASC d48jy0Lr16F5YDGNmzXD3CuJ3TgIXHYNgG7aneQd/kiZJW9C5xsTwA8O5AVOd3TQjozc 4a+O5vF/Vg5iW8q7pNFEIKnJpbtr8NC5UE9TutSEgpmrbdiV1+f9fuWH2DnInx8kskcs 6jiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Z2M+gqqaQpn9FcBPJlH4AznMxN1PloeM6UtHP44geQY=; b=mwZnvLGeqfsmqruTGifJMqvwYSzgJvVEbM5GZU6t9Enq+JTlTbO30k4ghkHfl4X0fl JtQpez5AEqeD/WAkji4kxxX/t0sxTWB3ZutU+TjmLlQEtYJ1ZaENBj9s4cl6GLHbHxD9 gdwTCCh2okuDbi4vl40gwAzK3odVor+nV0C4heNy49xV1tXdh2F6izJLWeDfTLSzFXmw 8Fc1lXIY3q8N2vFoNh1zvZ1OqhcENkkShla+9nnTBtzYY0e5CU1c8EdBLAFt5DftEzwq LfcX32gn1DXiNoUn1e2EbkDM/kxw7ALrgSpLXZNB/U61U3r61yl7Sc9s+e8qMFg0ArAN Wkpg== X-Gm-Message-State: AMCzsaURnwvQlRsAVl+NiRaQu5rziCTSriyoND6TSWaJZcD0knznq95K QQ/wiUxvIPX2pezOIEcA7ploP/I9xAszdYlWBQWo5g== X-Google-Smtp-Source: ABhQp+TNP5g+kk4qLBERGgjwzMjrMZl2vo4Uo1rto9boPf5zVP1iTFcIGMZm7cIGAWlHcye0mBvaQoa9Jp9qU7wMKXA= X-Received: by 10.157.51.84 with SMTP id u20mr2236626otd.9.1508974318011; Wed, 25 Oct 2017 16:31:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.32.78 with HTTP; Wed, 25 Oct 2017 16:31:57 -0700 (PDT) In-Reply-To: References: <2471941B-5297-465C-9658-9F8C775D77E6@gmail.com> From: "Owen O'Malley" Date: Wed, 25 Oct 2017 16:31:57 -0700 Message-ID: Subject: Re: Serde moved? version 2.3.0 To: "user@hive.apache.org" Content-Type: multipart/alternative; boundary="001a113e359c334a45055c6776ca" archived-at: Wed, 25 Oct 2017 23:32:09 -0000 --001a113e359c334a45055c6776ca Content-Type: text/plain; charset="UTF-8" On Wed, Oct 25, 2017 at 3:20 PM, Stephen Sprague wrote: > i see. interesting. i think this breaks a ton of opensource Serde's we've > all downloaded off the internet and have been using for years. openx json > serde and ibm xml serde come to mind. does this change render all these > incompatible now with 2.3.0? > Unfortunately, yes. With Hive 2.2.0, the SerDe class is still there. It was removed in 2.3.0. I suspect the developer didn't realize there were plugins that would break. > > I mean all i done in the past is download the jar file and put it in the > lib (or auxlib) dir and i was good to go. That clearly isn't going to fly > anymore then. > > hmmm. again i'm not a java weenie so rewriting java code isn't something > i would be proficient at. But it sounds like from a devops standpoint > there's no magic jar i can put in the class path to make these work? that > kinda is a deal-breaker to upgrade then. > Unfortunately, not. Looking at the change, the code now requires the user's serde to implement AbstractSerDe. If it doesn't, it will break. Effectively, the choices seem to be: 1. Change the plugin and recompile it. 2. Upgrade to Hive 2.2 instead of 2.3. 3. Make a case for reverting the change. I'm not sure what the original motivation of the change was. It seems like it was effectively a clean up. .. Owen > Am i interpreting this correctly? > > On Wed, Oct 25, 2017 at 2:31 PM, Owen O'Malley > wrote: > >> SerDe was removed by https://issues.apache.org/jira/browse/HIVE-15167 >> >> You should use AbstractSerDe instead. >> >> .. Owen >> >> On Oct 25, 2017, at 2:18 PM, Stephen Sprague wrote: >> >> hey guys, >> >> could be a dumb question but not being a java type of guy i'm not quite >> sure about it. I'm upgrading from 2.1.0 to 2.3.0 and encountering this >> error: >> >> class not found: org/apache/hadoop/hive/serde2/SerDe >> >> so in hive 2.1.0 i see it in this jar: >> >> * hive-serde-2.1.0.jar >> org/apache/hadoop/hive/serde2/SerDe.class >> >> >> but in hive 2.3.0 i don't see it in hive-serde-2.3.0.jar. >> >> >> so i ask where did it go in version 2.3.0? >> >> thanks, >> Stephen >> >> >> > --001a113e359c334a45055c6776ca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

= On Wed, Oct 25, 2017 at 3:20 PM, Stephen Sprague <spragues@gmail.com&= gt; wrote:
i see. interesting.=C2=A0 i thin= k this breaks a ton of opensource Serde's we've all downloaded off = the internet and have been using for years.=C2=A0 openx json serde and ibm = xml serde come to mind. does this change render all these incompatible now = with 2.3.0?=C2=A0=C2=A0

Un= fortunately, yes. With Hive 2.2.0, the SerDe class is still there. It was r= emoved in 2.3.0. I suspect the developer didn't realize there were plug= ins that would break.
=C2=A0
=

I mean all i done in the past is download the= jar file and put it in the lib (or auxlib) dir and i was good to go.=C2=A0= That clearly isn't going to fly anymore then.

hmmm.=C2=A0 again i'm not a java weenie so rewriting= java code isn't something i would be proficient at. But it sounds like= from a devops standpoint there's no magic jar i can put in the class p= ath to make these work?=C2=A0 that kinda is a deal-breaker to upgrade then.=

Unfortunately, not. Looking at= the change, the code now requires the user's serde to implement Abstra= ctSerDe. If it doesn't, it will break.=C2=A0

E= ffectively, the choices seem to be:
  1. Change the plugin and= recompile it.
  2. Upgrade to Hive 2.2 instead of 2.3.
  3. Make= a case for reverting the change. I'm not sure what the original motiva= tion of the change was. It seems like it was effectively a clean up.
.. Owen


<= div style=3D"font-family:courier new,monospace">Am i interpreting this corr= ectly?

On Wed, Oct 25, 2017 at 2:31 P= M, Owen O'Malley <owen.omalley@gmail.com> wrote:
SerDe w= as removed by=C2=A0https://issues.apache.org/jira/browse/HIVE-1516= 7

You should use AbstractSerDe instead.

.. Owen

On Oct 25, 2017, at 2:18 PM,= Stephen Sprague <spragues@gmail.com> wrote:

hey guys,

could be a dumb question but not being a java type of guy = i'm not quite sure about it. I'm upgrading from 2.1.0 to 2.3.0 and = encountering this error:

class no= t found: org/apache/hadoop/hive/serde2/SerDe

so in hive 2.1.0 i see it in this jar:

* hive-serde-2.1.0.jar
org/apache/hadoop/hive/ser= de2/SerDe.class
=

but in hive 2.3.0 i don't se= e it in hive-serde-2.3.0.jar.


so i ask where did it g= o in version 2.3.0?
<= br>
thanks,
Stephen


--001a113e359c334a45055c6776ca--