hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <owen.omal...@gmail.com>
Subject Re: Serde moved? version 2.3.0
Date Wed, 25 Oct 2017 23:31:57 GMT
On Wed, Oct 25, 2017 at 3:20 PM, Stephen Sprague <spragues@gmail.com> wrote:

> i see. interesting.  i think this breaks a ton of opensource Serde's we've
> all downloaded off the internet and have been using for years.  openx json
> serde and ibm xml serde come to mind. does this change render all these
> incompatible now with 2.3.0?
>

Unfortunately, yes. With Hive 2.2.0, the SerDe class is still there. It was
removed in 2.3.0. I suspect the developer didn't realize there were plugins
that would break.


>
> I mean all i done in the past is download the jar file and put it in the
> lib (or auxlib) dir and i was good to go.  That clearly isn't going to fly
> anymore then.
>
> hmmm.  again i'm not a java weenie so rewriting java code isn't something
> i would be proficient at. But it sounds like from a devops standpoint
> there's no magic jar i can put in the class path to make these work?  that
> kinda is a deal-breaker to upgrade then.
>

Unfortunately, not. Looking at the change, the code now requires the user's
serde to implement AbstractSerDe. If it doesn't, it will break.

Effectively, the choices seem to be:

   1. Change the plugin and recompile it.
   2. Upgrade to Hive 2.2 instead of 2.3.
   3. Make a case for reverting the change. I'm not sure what the original
   motivation of the change was. It seems like it was effectively a clean up.

.. Owen


> Am i interpreting this correctly?
>
> On Wed, Oct 25, 2017 at 2:31 PM, Owen O'Malley <owen.omalley@gmail.com>
> wrote:
>
>> SerDe was removed by https://issues.apache.org/jira/browse/HIVE-15167
>>
>> You should use AbstractSerDe instead.
>>
>> .. Owen
>>
>> On Oct 25, 2017, at 2:18 PM, Stephen Sprague <spragues@gmail.com> wrote:
>>
>> hey guys,
>>
>> could be a dumb question but not being a java type of guy i'm not quite
>> sure about it. I'm upgrading from 2.1.0 to 2.3.0 and encountering this
>> error:
>>
>> class not found: org/apache/hadoop/hive/serde2/SerDe
>>
>> so in hive 2.1.0 i see it in this jar:
>>
>> * hive-serde-2.1.0.jar
>> org/apache/hadoop/hive/serde2/SerDe.class
>>
>>
>> but in hive 2.3.0 i don't see it in hive-serde-2.3.0.jar.
>>
>>
>> so i ask where did it go in version 2.3.0?
>>
>> thanks,
>> Stephen
>>
>>
>>
>

Mime
View raw message