hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hongyuan Li (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14623) KafkaSink#init should set acks to 1,not 0
Date Wed, 05 Jul 2017 14:35:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074856#comment-16074856
] 

Hongyuan Li edited comment on HADOOP-14623 at 7/5/17 2:34 PM:
--------------------------------------------------------------

[~jojochuang] hard to write an only junit test to test it.

the infos about acks is from kafka document :
{code}
{{request.required.acks}} // using old Producer api or the version of kafka is less  than
0.9.x
or
{{acks}} // using new Producer api and kafka version more than 0.9.x

This value controls when a produce request is considered completed. Specifically, how many
other brokers must have committed the data to their log and acknowledged this to the leader?
Typical values are
0, which means that the producer never waits for an acknowledgement from the broker (the same
behavior as 0.7). This option provides the lowest latency but the weakest durability guarantees
(some data will be lost when a server fails).
1, which means that the producer gets an acknowledgement after the leader replica has received
the data. This option provides better durability as the client waits until the server acknowledges
the request as successful (only messages that were written to the now-dead leader but not
yet replicated will be lost).
-1, which means that the producer gets an acknowledgement after all in-sync replicas have
received the data. This option provides the best durability, we guarantee that no messages
will be lost as long as at least one in sync replica remains.
{code}

[DocumentationKafka 0.8.2|http://kafka.apache.org/082/documentation.html]
[Documentation Kafka 0.9.0|http://kafka.apache.org/090/documentation.html]
FROM the link below, if you use kafka below 0.9.x, should set  {{request.required.acks = 1}}
at least.When use new Producer above 0.9.x, should set {{acks = 1}} at least.



was (Author: hongyuan li):
[~jojochuang] hard to write an only junit test to test it.

the infos about acks is from kafka document :
{code}
request.required.acks	
This value controls when a produce request is considered completed. Specifically, how many
other brokers must have committed the data to their log and acknowledged this to the leader?
Typical values are
0, which means that the producer never waits for an acknowledgement from the broker (the same
behavior as 0.7). This option provides the lowest latency but the weakest durability guarantees
(some data will be lost when a server fails).
1, which means that the producer gets an acknowledgement after the leader replica has received
the data. This option provides better durability as the client waits until the server acknowledges
the request as successful (only messages that were written to the now-dead leader but not
yet replicated will be lost).
-1, which means that the producer gets an acknowledgement after all in-sync replicas have
received the data. This option provides the best durability, we guarantee that no messages
will be lost as long as at least one in sync replica remains.
{code}

[DocumentationKafka 0.8.2|http://kafka.apache.org/082/documentation.html]

FROM the link below, if you use kafka below 0.9.x, should set  request.required.acks = 1 at
least.When use new Producer above 0.9.x, should set acks = 1 at least.


> KafkaSink#init should set acks to 1,not 0
> -----------------------------------------
>
>                 Key: HADOOP-14623
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14623
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common, tools
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Hongyuan Li
>            Assignee: Hongyuan Li
>         Attachments: HADOOP-14623-001.patch
>
>
> {{KafkaSink}}#{{init}}  should set ack to *1* to make sure the message has been written
to the broker at least.
> current code list below:
> {code}
>   
>     props.put("request.required.acks", "0");
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message