pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2661) Pig uses an extra job for loading data in Pigmix L9
Date Mon, 03 Sep 2012 04:10:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447106#comment-13447106

Dmitriy V. Ryaboy commented on PIG-2661:

Looking at the code, I am not sure how you would make that work..

But perhaps one could emit an _extra tuple_ at the end instead of appending fields to a tuple?
Data about size, etc would then be contained in that last tuple (in a single field -- since
we may be joining a relation of one-field tuples).

This may still be problematic if type-specific tuples are generated and used for the loaders,
as the type of the last tuple won't match the schema of the rest of the tuples.

Considering all the options so far, I think it's still easiest to force a removal of the unnecessary
foreach in just the sampler job.
> Pig uses an extra job for loading data in Pigmix L9
> ---------------------------------------------------
>                 Key: PIG-2661
>                 URL: https://issues.apache.org/jira/browse/PIG-2661
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.9.0
>            Reporter: Jie Li
>            Assignee: Jie Li
>         Attachments: PIG-2661.0.patch, PIG-2661.1.patch, PIG-2661.2.patch, PIG-2661.3.patch,
PIG-2661.4.patch, PIG-2661.5.patch, PIG-2661.6.patch, PIG-2661.7.patch, PIG-2661.8.patch,
> See https://issues.apache.org/jira/browse/PIG-200?focusedCommentId=13260155&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13260155

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message