beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1249) Flatten with heterongeneous coders does not have a RunnableOnService test
Date Fri, 06 Jan 2017 18:54:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805287#comment-15805287
] 

Thomas Groh commented on BEAM-1249:
-----------------------------------

Resolution for runners is currently up to the runner.

There's no reason for the model to forbid disjoint coders - the contents of the output {{PCollection}}
should be encoded with the coder of that {{PCollection}}, and the contents of the input {{PCollections}}
can be decoded with their coders, as in any other {{PTransform}}. Depending on how the Runner
handles Flatten, this can be anywhere from a no-op (if the inputs to the flatten do not need
to be decoded nor outputs immediately encoded) to a decode-and-encode if the inputs are and
output should be materialized.

> Flatten with heterongeneous coders does not have a RunnableOnService test
> -------------------------------------------------------------------------
>
>                 Key: BEAM-1249
>                 URL: https://issues.apache.org/jira/browse/BEAM-1249
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Kenneth Knowles
>            Assignee: Thomas Groh
>            Priority: Minor
>             Fix For: 0.5.0
>
>
> Because there is no such test, we do not know which runners support this. It is perfectly
acceptable in the model as long as the types line up (including nullability) though in general
might incur a needless serde so also reasonable to just not support it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message