hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-911) [Piggybank] SequenceFileLoader
Date Wed, 12 Aug 2009 06:55:14 GMT

    [ https://issues.apache.org/jira/browse/PIG-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742239#action_12742239
] 

Alan Gates commented on PIG-911:
--------------------------------

Dmitry,

First this is great.  We've had requests to read Sequence files.  Being able to write them
also would be great.

A few thoughts:

1) This should not extend UTF8StorageConverter.  This loader will be returning actual data
types, not bytes that need to be interpreted.  I would think instead that it should implement
the bytesToX() methods itself and just throw an exception saying it didn't expect to do any
conversion.

2) The getSampledTuple looks fine if skip is handling getting the stream to the point that
reading the next tuple is viable.

3) In the bindTo call, where you obtain the key and value by reflection, should there be a
try/catch block there in case the cast to Writable fails?  In the same way, in describe schema
you're asking how to suppress warnings from the cast in reader.getKeyClass().  But don't you
want to check that what you got really is a writable, since there is no guarantee?



> [Piggybank] SequenceFileLoader 
> -------------------------------
>
>                 Key: PIG-911
>                 URL: https://issues.apache.org/jira/browse/PIG-911
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Dmitriy V. Ryaboy
>         Attachments: pig_sequencefile.patch
>
>
> The proposed piggybank contribution adds a SequenceFileLoader to the piggybank.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message