beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <>
Subject [jira] [Commented] (BEAM-1738) DataflowRunner should override Reshuffle transform
Date Fri, 17 Mar 2017 16:19:41 GMT


Thomas Groh commented on BEAM-1738:

Dataflow internally does no buffering and outputs elements with their original timestamps
when a ReshuffleTrigger is assigned to the GroupByKey. It doesn't need to reify the timestamps

The new tests that Reshuffle maintains timestamps should still apply.

> DataflowRunner should override Reshuffle transform
> --------------------------------------------------
>                 Key: BEAM-1738
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>            Reporter: Ben Chambers
>            Assignee: Thomas Groh
> Verify that the code works, and then remove the reification of windows for the Dataflow
Runner since it handles Reshufle specially and doesn't need the explicit reification.

This message was sent by Atlassian JIRA

View raw message