beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-2902) Add user state support for ParDo.Multi for the Dataflow runner
Date Mon, 11 Sep 2017 20:08:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luke Cwik updated BEAM-2902:
----------------------------
    Description: 
Add support for user state in a ParDo.Multi by:
* adding expansion/conversion logic within Dataflow service to add any needed GBKs to allow
expansion.

*OR*

* Add support for expansion inside the Beam SDK once the PTransformMatcher exposes a way to
know when the replacement is not required by checking that the preceding ParDos to a GBK are
key preserving.

  was:
Add support for user state in a ParDo.Multi by:
* adding expansion/conversion logic within Dataflow service to add any needed GBKs to allow
expansion.
OR
* Add support for expansion inside the Beam SDK once the PTransformMatcher exposes a way to
know when the replacement is not required by checking that the preceding ParDos to a GBK are
key preserving.


> Add user state support for ParDo.Multi for the Dataflow runner
> --------------------------------------------------------------
>
>                 Key: BEAM-2902
>                 URL: https://issues.apache.org/jira/browse/BEAM-2902
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-dataflow
>            Reporter: Luke Cwik
>            Priority: Minor
>              Labels: portability
>
> Add support for user state in a ParDo.Multi by:
> * adding expansion/conversion logic within Dataflow service to add any needed GBKs to
allow expansion.
> *OR*
> * Add support for expansion inside the Beam SDK once the PTransformMatcher exposes a
way to know when the replacement is not required by checking that the preceding ParDos to
a GBK are key preserving.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message