hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hongyuan Li (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14623) fixed some bugs in KafkaSink
Date Sat, 08 Jul 2017 08:07:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079033#comment-16079033
] 

Hongyuan Li edited comment on HADOOP-14623 at 7/8/17 8:06 AM:
--------------------------------------------------------------

I highly recommend Two points:
1、should use {{acks}} = {{1}}.
2、update kafka client version to {{0.10.1.0}},  which has a IntegerSerializer class If kafka
sink want to generate a kafka producer with the the type of key being Integer.
The last patch will fix the two, If you don't think so, close the jira. Thanks for any advice.


was (Author: hongyuan li):
I highly recommend Two points:
1、should use acks = 1.
2、update kafka client version to 0.10.1. which has a IntegerSerializer class If kafka sink
want to generate a kafka producer with the the type of key being Integer.
The last patch will fix the two, If you don't think so, close the jira.Thanks.

> fixed some bugs in KafkaSink 
> -----------------------------
>
>                 Key: HADOOP-14623
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14623
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common, tools
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Hongyuan Li
>            Assignee: Hongyuan Li
>         Attachments: HADOOP-14623-001.patch
>
>
> {{KafkaSink}}#{{init}}  should set ack to *1* to make sure the message has been written
to the broker at least.
> current code list below:
> {code}
>   
>     props.put("request.required.acks", "0");
> {code}
> *Update*
> find another bug about this class, {{key.serializer}} used {{org.apache.kafka.common.serialization.ByteArraySerializer}},
however, the key properties of Producer is Integer, codes list below:
> {code}
>     props.put("key.serializer",
>         "org.apache.kafka.common.serialization.ByteArraySerializer");
> ……………
>  producer = new KafkaProducer<Integer, byte[]>(props);
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message