beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-1314) DoFn per-key lifecycle (Setup/Teardown)
Date Wed, 25 Jan 2017 19:38:27 GMT
Eugene Kirpichov created BEAM-1314:
--------------------------------------

             Summary: DoFn per-key lifecycle (Setup/Teardown)
                 Key: BEAM-1314
                 URL: https://issues.apache.org/jira/browse/BEAM-1314
             Project: Beam
          Issue Type: Improvement
          Components: beam-model
            Reporter: Eugene Kirpichov
            Assignee: Frances Perry


DoFn's that use state and timers are implicitly per-key. Setup/Teardown methods are usually
used to establish expensive resources - long-standing connections and such.

For per-key DoFn's, we'd often want to use these per-key, so it'd be good to have ability
in the model to ask that there be 1 instance of the DoFn per key, reused between e.g. different
timer or trigger firings for this key, but not used for other keys.

E.g. this would be particularly useful for Splittable DoFn - for its ability to reuse expensive
resources between checkpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message