hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <owen.omal...@gmail.com>
Subject Re: Serde moved? version 2.3.0
Date Fri, 27 Oct 2017 20:41:53 GMT
Ok, I've created a jira to revert the change.

*https://issues.apache.org/jira/browse/HIVE-17924
<https://issues.apache.org/jira/browse/HIVE-17924>*

On Wed, Oct 25, 2017 at 7:37 PM, Stephen Sprague <spragues@gmail.com> wrote:

> thanks guys for the clarification.  i'll work on perhaps finding the
> source code and re-implementing if i can.  good to know.
>
>
>
> On Wed, Oct 25, 2017 at 6:32 PM, Owen O'Malley <owen.omalley@gmail.com>
> wrote:
>
>> I considered that, but it won't work.
>>
>> The Hive 2.2 code looks like:
>>
>> public interface SerDe { ... }
>> public abstract class AbstractSerDe implements SerDe { ... }
>>
>> ... a lot of code using SerDe ...
>>
>> The Hive 2.3 code looks like:
>>
>> public abstract class AbstractSerDe { .... }
>>
>> ... a lot of code using AbstractSerDe ...
>>
>> You could create a SerDe interface to avoid the ClassNotFound exception,
>> but I assume the external classes do:
>>
>> public FooSerDe implements SerDe { ... }
>>
>> when Hive 2.3 tries to use the instance as an AbstractSerDe, it will
>> fail.
>>
>> With SerDe being an interface and AbstractSerDe being an abstract class,
>> we also can't play games with inverting the class hierarchy.
>>
>> .. Owen
>>
>> On Wed, Oct 25, 2017 at 4:38 PM, Matt Burgess <mattyb149@gmail.com>
>> wrote:
>>
>>> Perhaps a fourth option is a “bridge” JAR, to implement the original
>>> class by subclassing AbstractSerDe, unless the API has changed such that
>>> such a mapping cannot be done.
>>>
>>> Regards,
>>> Matt
>>>
>>>
>>>
>>> On Oct 25, 2017, at 7:31 PM, Owen O'Malley <owen.omalley@gmail.com>
>>> wrote:
>>>
>>>
>>> On Wed, Oct 25, 2017 at 3:20 PM, Stephen Sprague <spragues@gmail.com>
>>> wrote:
>>>
>>>> i see. interesting.  i think this breaks a ton of opensource Serde's
>>>> we've all downloaded off the internet and have been using for years.  openx
>>>> json serde and ibm xml serde come to mind. does this change render all
>>>> these incompatible now with 2.3.0?
>>>>
>>>
>>> Unfortunately, yes. With Hive 2.2.0, the SerDe class is still there. It
>>> was removed in 2.3.0. I suspect the developer didn't realize there were
>>> plugins that would break.
>>>
>>>
>>>>
>>>> I mean all i done in the past is download the jar file and put it in
>>>> the lib (or auxlib) dir and i was good to go.  That clearly isn't going to
>>>> fly anymore then.
>>>>
>>>> hmmm.  again i'm not a java weenie so rewriting java code isn't
>>>> something i would be proficient at. But it sounds like from a devops
>>>> standpoint there's no magic jar i can put in the class path to make these
>>>> work?  that kinda is a deal-breaker to upgrade then.
>>>>
>>>
>>> Unfortunately, not. Looking at the change, the code now requires the
>>> user's serde to implement AbstractSerDe. If it doesn't, it will break.
>>>
>>> Effectively, the choices seem to be:
>>>
>>>    1. Change the plugin and recompile it.
>>>    2. Upgrade to Hive 2.2 instead of 2.3.
>>>    3. Make a case for reverting the change. I'm not sure what the
>>>    original motivation of the change was. It seems like it was effectively a
>>>    clean up.
>>>
>>> .. Owen
>>>
>>>
>>>> Am i interpreting this correctly?
>>>>
>>>> On Wed, Oct 25, 2017 at 2:31 PM, Owen O'Malley <owen.omalley@gmail.com>
>>>> wrote:
>>>>
>>>>> SerDe was removed by https://issues.apache.org/jira/browse/HIVE-15167
>>>>>
>>>>> You should use AbstractSerDe instead.
>>>>>
>>>>> .. Owen
>>>>>
>>>>> On Oct 25, 2017, at 2:18 PM, Stephen Sprague <spragues@gmail.com>
>>>>> wrote:
>>>>>
>>>>> hey guys,
>>>>>
>>>>> could be a dumb question but not being a java type of guy i'm not
>>>>> quite sure about it. I'm upgrading from 2.1.0 to 2.3.0 and encountering
>>>>> this error:
>>>>>
>>>>> class not found: org/apache/hadoop/hive/serde2/SerDe
>>>>>
>>>>> so in hive 2.1.0 i see it in this jar:
>>>>>
>>>>> * hive-serde-2.1.0.jar
>>>>> org/apache/hadoop/hive/serde2/SerDe.class
>>>>>
>>>>>
>>>>> but in hive 2.3.0 i don't see it in hive-serde-2.3.0.jar.
>>>>>
>>>>>
>>>>> so i ask where did it go in version 2.3.0?
>>>>>
>>>>> thanks,
>>>>> Stephen
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message