pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Wagner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3571) Early termination when processing limit
Date Wed, 13 Nov 2013 19:17:20 GMT

    [ https://issues.apache.org/jira/browse/PIG-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821697#comment-13821697

Mark Wagner commented on PIG-3571:

This would be a nice optimization for us to have. Have you thought about how to handle multiqueries?
I think this ties back to our conversation about EOP and your comment that in MR, EOP means
both end of key and real end of processing. Should we separate those two?

> Early termination when processing limit
> ---------------------------------------
>                 Key: PIG-3571
>                 URL: https://issues.apache.org/jira/browse/PIG-3571
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.13.0
>         Attachments: PIG-3571-0.patch
> When we pull enough records for POLimit, currently we still pull the whole pipeline for
every key. We could use a flag to avoid that.

This message was sent by Atlassian JIRA

View raw message