beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-2704) KafkaIO: NPE without key serializer set
Date Thu, 26 Oct 2017 18:59:01 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Raghu Angadi updated BEAM-2704:
-------------------------------
    Fix Version/s: 2.2.0

> KafkaIO: NPE without key serializer set
> ---------------------------------------
>
>                 Key: BEAM-2704
>                 URL: https://issues.apache.org/jira/browse/BEAM-2704
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>            Reporter: Chris Pettitt
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 2.2.0
>
>
> The KafkaIO javadoc implies that you do not need to set a Serializer if you only want
to emit values:
> {code}
>  * <p>Often you might want to write just values without any keys to Kafka. Use
{@code values()} to
>  * write records with default empty(null) key:
>  *
>  * <pre>{@code
>  *  PCollection<String> strings = ...;
>  *  strings.apply(KafkaIO.<Void, String>write()
>  *      .withBootstrapServers("broker_1:9092,broker_2:9092")
>  *      .withTopic("results")
>  *      .withValueSerializer(new StringSerializer()) // just need serializer for value
>  *      .values()
>  *    );
>  * }</pre>
> {code}
> However, if you don't set the key serializer then Kafka blows up when trying to instantiate
the key serializer (in Kafka 0.10.1, at least). It would be more convenient if KafkaIO worked
as documented and assigned a null serializer if values() is used.  
> Relevant stack trace:
> {code}
> Caused by: java.lang.NullPointerException
> 	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:230)
> 	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:163)
> 	at org.apache.beam.sdk.io.kafka.KafkaIO$KafkaWriter.setup(KafkaIO.java:1582)
> 	at org.apache.beam.sdk.io.kafka.KafkaIO$KafkaWriter$DoFnInvoker.invokeSetup(Unknown
Source)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message