pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2661) Pig uses an extra job for loading data in Pigmix L9
Date Mon, 03 Sep 2012 02:42:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447091#comment-13447091
] 

Jie Li commented on PIG-2661:
-----------------------------

Nice analysis!

As the foreach may or may not exist in the query, playing with it may be tricky. Maybe we
can change the current way of passing the internal info from PoissonSampleLoader to the GetMemNumRows.
What we need is to tag each tuple with an integer needed by the GetMemNumRows. Currently we
simply append the tag to the tuple's data, which can confuse other operators. A more general
method is to add a Map field in the tuple so operators in the pipeline can add into the Map
whatever tags they want without interrupting the normal data processing.

What do you think?
                
> Pig uses an extra job for loading data in Pigmix L9
> ---------------------------------------------------
>
>                 Key: PIG-2661
>                 URL: https://issues.apache.org/jira/browse/PIG-2661
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.9.0
>            Reporter: Jie Li
>            Assignee: Jie Li
>         Attachments: PIG-2661.0.patch, PIG-2661.1.patch, PIG-2661.2.patch, PIG-2661.3.patch,
PIG-2661.4.patch, PIG-2661.5.patch, PIG-2661.6.patch, PIG-2661.7.patch, PIG-2661.8.patch,
PIG-2661.plan.txt
>
>
> See https://issues.apache.org/jira/browse/PIG-200?focusedCommentId=13260155&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13260155

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message