hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-911) [Piggybank] SequenceFileLoader
Date Tue, 18 Aug 2009 03:07:14 GMT

    [ https://issues.apache.org/jira/browse/PIG-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744343#action_12744343

Dmitriy V. Ryaboy commented on PIG-911:

Concerning making this a StoreFunc, as well -- the StoreFunc interface is not very friendly
to this.
All you get in the bind call is the output stream; for LoadFunc, you also get the name of
the file (or, presumably, whatever it was the user passed in under the guise of a file name).
 This means that for the LoadFunc, I was able to use the passed in filename to back into a
Path and a FileSystem.  I can't do the same for StoreFunc, where the filename is not available
-- only the output stream is.  That means I can't create the appropriate SequenceFile.Writer
.  Is there a way around this limitation that does not involve requiring special constructor
parameters to be used?  
Is it possible to change the StoreFunc api to provide this information, or to make it available
through some side channel (MapRedUtils or similar)?

> [Piggybank] SequenceFileLoader 
> -------------------------------
>                 Key: PIG-911
>                 URL: https://issues.apache.org/jira/browse/PIG-911
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Dmitriy V. Ryaboy
>         Attachments: pig_911.2.patch, pig_sequencefile.patch
> The proposed piggybank contribution adds a SequenceFileLoader to the piggybank.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message