hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Ryaboy <dvrya...@gmail.com>
Subject Re: Caster interface and byte conversion
Date Tue, 24 Aug 2010 20:22:49 GMT
As far as the toBytes methods -- I am not sure what they were originally
for. They aren't actually called anywhere that I can find, except my new
HBase stuff.
You are right, I could make it two interfaces, but I consolidated them for
simplicity of use/implementation. Now that I think about it, I can put all
the methods into StoreCaster and just have a unioning interface for

public interface LoadStoreCaster extends LoadCaster, StoreCaster {


Does that seem ok?


On Tue, Aug 24, 2010 at 10:01 AM, Alan Gates <gates@yahoo-inc.com> wrote:

> One other comment.  By making this part of an interface that extends
> LoadCaster you are assuming the implementing class is both a load and store
> function.  It makes more sense to have a separate StoreCaster interface
> rather than extending LoadCaster.
> Alan.
> On Aug 24, 2010, at 9:18 AM, Alan Gates wrote:
>  This seems fine.  Is the Pig engine at any point testing to see if the
>> interface is implemented and if so calling toBytes, or is this totally
>> for use inside the store functions themselves to serialize Pig data
>> types?
>> Alan.
>> On Aug 22, 2010, at 1:40 AM, Dmitriy Ryaboy wrote:
>>  The current HBase patch on PIG-1205 (patch 7) includes this
>>> refactoring.
>>> Please take a look if you have concerns.
>>> Or just if you feel like reviewing the code... :)
>>> -D
>>> On Sat, Aug 21, 2010 at 5:22 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
>>> wrote:
>>>  I just noticed that even though Utf8StorageConverter implements the
>>>> various
>>>> byte[] toBytes(Obj o) methods, they are not part of the LoadCaster
>>>> interface
>>>> -- and therefore can't be relied on when using modular Casters, like I
>>>> am
>>>> trying to do for the HBaseLoader.
>>>> Since we don't want to introduce backwards-incompatible changes, I
>>>> propose
>>>> adding a ByteCaster interface that defines these methods, and
>>>> extending
>>>> Utf8StorageConverter to implement them (without actually changing the
>>>> implementation at all).
>>>> That way StoreFuncs that need to convert to bytes can use pluggable
>>>> converters. Objections?
>>>> -D

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message