beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <>
Subject [jira] [Created] (BEAM-1611) Stateful ParDo broken in Dataflow batch
Date Thu, 02 Mar 2017 23:18:45 GMT
Kenneth Knowles created BEAM-1611:

             Summary: Stateful ParDo broken in Dataflow batch
                 Key: BEAM-1611
             Project: Beam
          Issue Type: Bug
          Components: runner-dataflow
            Reporter: Kenneth Knowles
            Assignee: Kenneth Knowles
            Priority: Critical

The needed shuffle is not performed in batch mode. Confirmed that it is performed in streaming.

Filing here as the quickest fix is to work around with transform overrides in Beam, analogous
to those in the direct runner.

This message was sent by Atlassian JIRA

View raw message