beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1738) DataflowRunner should override Reshuffle transform
Date Fri, 17 Mar 2017 16:19:41 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930236#comment-15930236
] 

Thomas Groh commented on BEAM-1738:
-----------------------------------

Dataflow internally does no buffering and outputs elements with their original timestamps
when a ReshuffleTrigger is assigned to the GroupByKey. It doesn't need to reify the timestamps
explicitly.

The new tests that Reshuffle maintains timestamps should still apply.

> DataflowRunner should override Reshuffle transform
> --------------------------------------------------
>
>                 Key: BEAM-1738
>                 URL: https://issues.apache.org/jira/browse/BEAM-1738
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>            Reporter: Ben Chambers
>            Assignee: Thomas Groh
>
> Verify that the code works, and then remove the reification of windows for the Dataflow
Runner since it handles Reshufle specially and doesn't need the explicit reification.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message