beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason White (JIRA)" <>
Subject [jira] [Commented] (BEAM-440) Create.values() returns a type-unsafe Coder
Date Tue, 16 Aug 2016 23:59:21 GMT


Jason White commented on BEAM-440:

I tried tackling this issue as an onboarding issue. If I'm understanding this correctly, since
the coder is determined from the TypeDescriptor only at runtime, it wouldn't be very pretty
to add the TypeDescriptor to the Values class just for this purpose.

Instead, I propose adding a Create#empty(Coder) method to explicitly create an empty PCollection
using the specified Coder. This should provide an implicit suggestion to not rely on Create.of()
to create an empty PCollection.

> Create.values() returns a type-unsafe Coder
> -------------------------------------------
>                 Key: BEAM-440
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Daniel Halperin
>            Assignee: Manu Zhang
>              Labels: newbie, starter
> Create.values() with no arguments will default to a VoidCoder, unless one is set later
with #setCoder(Coder).
> Although it will encode its input correctly, this seems like a bad choice in many cases.
E.g., with Flatten:
>     PCollection<KV<SomeClass, Integer>> initial = p.apply("First", Create.<KV<SomeClass,
>     PCollection<KV<SomeClass, Integer>> second =
>         p.apply("Second", Create.of("a", "b")).apply(ParDo.of(new MyAvroDoFn()));
>     PCollectionList
>     .of(initial).and(second)
>         .apply(Flatten.<KV<SomeClass, Integer>>pCollections());
> This crashes trying to cast a KV from "Second" to a Void.class.
> 1. Suggest throwing a warning in #getDefaultOutputCoder when defaulting to VoidCoder
for an empty elements list. Should this be an error?
> 2. Suggest adding something like Create.empty(TypeDescriptor) to handle this case properly.

This message was sent by Atlassian JIRA

View raw message