flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1656) Filtered Semantic Properties for Operators with Iterators
Date Tue, 24 Mar 2015 14:39:52 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377965#comment-14377965

ASF GitHub Bot commented on FLINK-1656:

Github user fhueske commented on the pull request:

    Do you think this rule is "easy enough" for users? 
    It would make the handling consistent for all group-wise operators.

> Filtered Semantic Properties for Operators with Iterators
> ---------------------------------------------------------
>                 Key: FLINK-1656
>                 URL: https://issues.apache.org/jira/browse/FLINK-1656
>             Project: Flink
>          Issue Type: Bug
>          Components: Documentation
>    Affects Versions: 0.9
>            Reporter: Fabian Hueske
>            Assignee: Fabian Hueske
>            Priority: Critical
> The documentation of ForwardedFields is incomplete for operators with iterator inputs
(GroupReduce, CoGroup). 
> This should be fixed ASAP, because it can lead to incorrect program execution.
> The conditions for forwarded fields on operators with iterator input are:
> 1) forwarded fields must be emitted in the order in which they are received through the
> 2) all forwarded fields of a record must stick together, i.e., if your function builds
record from field 0 of the 1st, 3rd, 5th, ... and field 1 of the 2nd, 4th, ... record coming
through the iterator, these are not valid forwarded fields.
> 3) it is OK to completely filter out records coming through the iterator.
> The reason for these conditions is that the optimizer uses forwarded fields to reason
about physical data properties such as order and grouping. Mixing up the order of records
or emitting records which are composed from different input records, might destroy a (secondary)
order or grouping.

This message was sent by Atlassian JIRA

View raw message