hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerrit Jansen van Vuuren (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1237) Piggybank MutliStorage - specify field to write in output
Date Fri, 12 Feb 2010 14:20:27 GMT

    [ https://issues.apache.org/jira/browse/PIG-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833004#action_12833004
] 

Gerrit Jansen van Vuuren commented on PIG-1237:
-----------------------------------------------

I forgot to mention that I've also added some logic to replace any '/' characters with '_'
these might be in the field used to select the output file names.
The reason for doing this is that we had this field as a file name of the original source
from where our logs came from. Without this change the files were written
to directories like /usr/share/tomcat5/logs/mylog.log instead of [outputdir]/mylog.log or
[outputdir]/usr_share_tomcat4_logs_mylog.log


> Piggybank MutliStorage - specify field to write in output
> ---------------------------------------------------------
>
>                 Key: PIG-1237
>                 URL: https://issues.apache.org/jira/browse/PIG-1237
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Gerrit Jansen van Vuuren
>            Assignee: Gerrit Jansen van Vuuren
>            Priority: Minor
>         Attachments: PIG-1237.patch
>
>
> I've made a modification to the piggy bank MutliStorage class that allows to optionally
specify the index of the field in each tuple to write to output.
> This feature allows to have records with metadata like seqno, time of upload etc, and
then to combine files from these records into one but without the metadata.
> e.g. 
> 1: date type seq1 data
> 2:  date type seq2 data
> then write output grouped by type and ordered by sequence:
> data
> data

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message