beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-800) DoFnLifeCycleManager should hold DoFnInvoker, not DoFn/OldDoFn
Date Mon, 24 Oct 2016 18:42:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602848#comment-15602848
] 

Kenneth Knowles commented on BEAM-800:
--------------------------------------

The porting of the DirectRunner occured on [#1157|https://github.com/apache/incubator-beam/pull/1157],
with lots of little fixes to make it all ready. The commit that actually ported the runner
is [1919d8b|https://github.com/apache/incubator-beam/pull/1157/commits/1919d8b3a850bd146137652546da851ee461cd28].
If you take a look, the {{DoFnLifecycleManager}} caches each {{DoFn}} instance, and tears
it down, but the repeated instantiation of a {{DoFnInvoker}} for it is redundant. It is possible
that even the {{DoFnRunner}} could be cached here (a {{DoFnRunner}} is essentially a {{DoFnInvoker}}
fixed to one particular context).

> DoFnLifeCycleManager should hold DoFnInvoker, not DoFn/OldDoFn
> --------------------------------------------------------------
>
>                 Key: BEAM-800
>                 URL: https://issues.apache.org/jira/browse/BEAM-800
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-direct
>            Reporter: Kenneth Knowles
>            Priority: Minor
>
> The {{DirectRunner}}'s support class {{DoFnLifecycleManager}} holds a cache of deserialized
{{OldDoFn}} s, now being ported to {{DoFn}} s. But to execute a {{DoFn}} there is another
layer of indirection through a {{DoFnInvoker}}, which is the best object to cache here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message