pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xianda Ke (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-4876) OutputConsumeIterator can't handle the last buffered tuples for some Operators
Date Tue, 19 Apr 2016 02:02:25 GMT
Xianda Ke created PIG-4876:
------------------------------

             Summary: OutputConsumeIterator can't handle the last buffered tuples for some
Operators
                 Key: PIG-4876
                 URL: https://issues.apache.org/jira/browse/PIG-4876
             Project: Pig
          Issue Type: Sub-task
            Reporter: Xianda Ke
            Assignee: Xianda Ke


Some Operators, such as MergeCogroup, Stream, CollectedGroup etc buffer some input records
to constitute the result tuples. The last result tuples are buffered in the operator.  These
Operators need a flag to indicate the end of input, so that they can flush and constitute
their last tuples.
Currently, the flag 'parentPlan.endOfAllInput' is targeted for flushing the buffered tuples
in MR mode.  But it does not work with OutputConsumeIterator in Spark mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message