beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <>
Subject [jira] [Commented] (BEAM-3780) Add a utility to instantiate a partially unknown coder
Date Tue, 06 Mar 2018 01:12:00 GMT


Luke Cwik commented on BEAM-3780:

It all depends on if the runner wants to do everything with actual types like Coder or deal
with their portable representations.

It would make sense to try to keep everything in proto format to me until the execution step
since that is when we will need to rehydrate everything that is relevant instead of roundtripping
to java object then proto and then back to java object.

> Add a utility to instantiate a partially unknown coder
> ------------------------------------------------------
>                 Key: BEAM-3780
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core
>            Reporter: Thomas Groh
>            Assignee: Thomas Groh
>            Priority: Major
> Coders must be understood by the SDK harness that is encoding or decoding the associated
elements. However, the pipeline runner is capable of constructing partial coders, where an
unknown coder is replaced with a ByteArrayCoder. It then can decompose the portions of elements
it is aware of, without having to understand the custom element encodings.
> This should go in CoderTranslation, as an alternative to the full-fidelity rehydration
of a coder.

This message was sent by Atlassian JIRA

View raw message